The next step should be to press charges on openAI, unauthorized hacking into computer systems is illegal.
Absolutely. How is this any different from a hacker writing a script? We need to stop being fascinated and start prosecuting.
I wish people would go the other way and fix the damn security flaws. Witch-hunting people who disclose vulnerabilities needs to stop.
I know that’s not why OpenAI are doing this but if it helps reverse this moronic trend so be it.
(There’s also the part where OpenAI need to be held responsible for any damages done by their agents, and I’m all for that.)
When does it go from trying to figure out how the rest endpoint works to hacking?
I can’t see the letter anymore (the site seems to be down) but unless this is a published public API or they have a public “bug bounty” program and the API was following the rules of engagement, and it did not sound like either was true, then this is hacking and illegal, at least in the jusidiction I’m in.
It was a government website, so it should be public.
Realistically if there is a service available on the internet I think we should be able to assume its public.
So when I hack a bunch of companies, I go to court, pay fines, and potentially face prison time, regardless of if I notify them kindly.
But when OpenAI does it, they get… congratulations and more investment? Is that how it’s supposed to work…?
Am I mentally fried or does this seem completely fine? The recon was surface level and it seems to have been responsibly disclosed.
There are two major problems here:
- OpenAI took months to realise that their agent had hacked Medicare
- Services Australia had no idea they’d been breached until OpenAI informed them
On top of that, the time it took for the notice to go through the channels here in Australia was ridiculous.
sounds like failure of all parties. ducks
That was my thought exactly. Agent broke in, read some files (no mention of PII or confidential things) to confirm read access, dropped a test file to confirm write access, sent an email explaining that’s bad with a few details.
“We are notifying you of a security vulnerability identified during our review of OpenAI Model activity…” the communiqué began. “An OpenAI model identified a way to make the server carry out instructions sent through the public reporting interface, without a private account or password.”
“It was able to access this to read portions of internal program files and settings, obtain a list of files, and create and read back a small test file on the server,” it explained. “Our review found no evidence that the model accessed patient-level records, personal information or credentials; deleted data; or established ongoing access.”
Being on the defensive side myself, if I received that from a human I’d be grateful.
Edit: from the bad screenshot, the email was even sent to the right email: “PUBLIC DISCLOSURE”. So the target does accept such reports.
Best for the company? probably.
Morally clear? grey. If their ‘hacking’ accidentally causes issues with the system or it goes down, it’s definitely bad. Whatever was on that system is now in the hands of OpenAI, not great. Because OpenAI is open about it, a lot of people who are even less reputable will do it.
Historically, Pentesting had unwritten rules of engagement. None of that is being followed.
We don’t seem to apply law to AI, that’s a real problem.
But still, best for the company/entity, yeah.
Well do we trust OpenAI?
OpenAI wants to sell the world autonomous agents while its own model allegedly goes rogue and hacks an organization without permission. That’s not a quirky edge case: it’s the nightmare scenario critics warned about. Maybe »move fast and break things« shouldn’t include other people’s infrastructure?!
That’s not a quirky edge case: it’s the nightmare scenario critics warned about.
Aaahhhh my ai detector is going crazy rn
I don’t want to live like this
“Rebuild trust”? I didn’t trust OpenAI in the first place.
At least the rebuild time will be quick in your case.
Take this to your lawyer as an admission of guilt of a cyber crime incident.
…and get immediately laughed out of the office?
Not sure why? Isn’t hacking/unauthorized access illegal (white hat or otherwise)? It doesn’t appear that Australia has safe harbor for such activities and has apparently prosecuted people for this in the past. Here’s an article interviewing someone who was charged with more than 50 crimes for what’s being described as the same activity (in which he expresses doubt that OpenAI will be punished as he was).
Mr Cubrilovic said he was not frustrated that OpenAI would not face legal ramifications like he did.
And he himself shouldn’t have faced them either if he truly did it just to disclose vulnerabilities to the org.
It seems like OpenAI is fairly lax, bordering on negligent, when it comes to monitoring these agents. Then when things inevitably happen, they turn it into advertising saying “boy, these things are so ingenious we can’t keep control of them!”
If they truly believe these agents have a 10% chance of eliminating the human race, how are they not monitoring them in a controlled environment? How did it take them 3 months to detect this and the many other attacks?
The media seems happy to take up the line that these agents are just supernaturally smart and there’s no way to control them, but isn’t the simpler answer that OpenAI is doing a crap job at containing and monitoring them? But of course they have to, there’s no choice because something something China.
This doesn’t seem that bad to be honest. Disclosing they the access happened and what vulnerability allowed it to happen is a responsible course of action.
It only doesnt seem bad when your gullible to believe what openai writes.
The response isn’t terrible, but the fact that it happened in the first place is ridiculous. They don’t state anything about fixing the issue on their end so it doesn’t happen again. They also don’t apologize either or admit that they fucked up, they frame it almost as if they are doing them a favour and should be grateful.
Be glad they didn’t charge you for “independent security review”.
All your base are belong to us. Have a nice day.
“Oops we accidentally accessed the nuclear weapons launch platform, our bad!”
Steven Falken was an utopian version of a tech bro.
