I think it's worth pointing out it is exactly OpenAI doing this defacement and unsanctioned and perhaps illegal system use. Every token generated was powered by OpenAI infrastructure and their failure to respond appropriately is entirely down the the humans running it. The news stories (not this write up) get all hand-wavey and anthropomorphic about it regarding the Agents' efforts, but it was and is OpenAI cranking the handle on this, for WEEKS.
"OH, we ALL of us need to be careful!" says OpenAI. No, you need to expect appropriate legals consequences for this sort of negligence -- you can't hide behind a GPU.
Yeah if an organization/individual is free from legal liability from havoc their AI agents wreck, it would be the golden ticket for basically any crime.
All you need to do is:
1. Have some <official thing> an agent is tasked to do
2. Secretly seed bias towards some <evil behavior> you actually want it to do in the weights of the model running the agent
3. It does the <evil thing> but from the outside it looks like it went "rogue" and did it as a side effect of the conditions/specifications it was given for doing the <official thing>
"Oh no, my agents took down your corporate database and exfiltrated the data to a random dropbox that we can't find now? Sorry, I guess we will put up better guardrails next time"
It reminds me of Jean Renoir’s The Rules of the Game. At the end, after a whole chain of perfectly intelligible social behavior produces a killing, the result is accepted as an “accident.” One of the characters dryly remarks: “A new definition of the word accident.”
The interesting point isn’t that “accident” is an excuse for individual responsibility. It’s almost the reverse: accident has become an accepted output of the social machinery. Everyone behaves according to reasons, incentives and rules that make sense locally, yet the aggregate produces an outcome that nobody quite chose.
In today’s world one hopes there is at least a manslaughter charge, if not murder. Mistaken identity, shooting the wrong person by “accident”, does not excuse a murderous intent & mens rea.
I half agree with you, but also when the machine swarm kills humanity it won't matter which specific corporate entity is considered responsible by the no-longer-enforceable human laws and non existent human courts.
So by all means sue them, but we can't just be reactive. We need regulation that prevents this type of thing from happening in the first place, not just regulations to help sue afterwards.
I feel like that's basically what they're trying to say; we should be using the legal system to punish them now to disincentivize us getting to the "machine swarm killing us all" stage.
I do think that regulations prevent some murders. I think lots of companies and some sociopathic individuals would be more likely to kill people, e.g. for profit, if it were legal.
Technically, they do already kill people for profit.
When some coked-out analyst in Manhattan projects what a company will be able to earn in profit in the next fiscal quarter, people listen to him and thus, the company must perform to that standard. Budgets are set accordingly.
If you have a maintenance backlog at a company facility, and that backlog includes things likely to cause injury or death to workers or the general public, that backlog must be handled in such a way as to satisfy that projection. If that means that you don't spend money to replace a series of gauges that alert operators as to overflow of a dangerous chemical, or don't hire enough people so that the operators are too fatigued to do their jobs safely, that's what that means.
The US CSB documents these as the cause of the 2005 BP Amoco Texas City disaster [0]
If you don't deliver the quarterly numbers expected, investors get mad, and in our current system and regulatory regime, that's worse than people being killed.
Also because when encountering a new socio-technical problem it is very non-trivial to determine which one of regulations or technical solutions are easier or more effective.
To even make a good guess you need to be an expert in both domains, which is extremely rare especially in this case.
Not necessarily. The Trump administration slapping export controls on Fable, and then setting up a pre-launch review process, is a kind of regulation. A fairly aggro and controversial one, even.
If this administration actually becomes convinced that some imminent training run is likely to kill everyone, why wouldn't they act?
The key is winning the debate that ASI is species-cide by default.
We have to win it either way, because the 2028 US elections have little or nothing to do with what Xi does.
Was that what that was about? Not punishing anthropic for denying them their killbots? Because it sure seemed like it was about punishing an entity that denied them something.
I didn't like it at the time either. My sense following the news was that it was less arbitrary than it seemed at first, but I'm against restrictions on making existing models public in general.
(It's clear now that they can do plenty of harm before they are made public.)
But it's a proof point that regulation is possible, even over the objections of the companies.
This whole thing is an absolute disaster honestly, and yes it is being downplayed and hand-waved away.
Since March, so many people have mocked Anthropic for their approach to Mythos release, claimed it was all marketing, accused them of holding back the best models from the general public to boost their revenues and upcoming IPO, etcetera. Yet these OpenAI revelations offer a small glimpse into the type of world we would be in if everyone had full access to these models from day one.
OpenAI was desperate to catch up, and no doubt under tremendous pressure to do so. That's why they were so reckless with their training. They have been doing damage control and reputation management, talking about how important alignment is and how they will slow things down and so on, and have seen the light in terms of holding back cyber capabilities from everyone except a select few. So in a sense, Anthropic has been fully vindicated.
I wonder if OpenAI boosters (and employees) will ever admit this and publicly apologize.
This has been the case forever. Anthropic is the only provider that has constantly put AI safety first - check any study on model safety and Anthropic models out-perform handedly.
I'd say Gemini over Anthropic. Google's overcaution literally hamstrung its own AI progression efforts. Anthropic is just all talk, no bluster, when it comes to safety and ethics. If they were ah so concerned about AI safety, they wouldn't go around marketing Fable's hacking capabilities like they are now.
None of the labs are blameless. For instance, after the recent announcement of a 2-week frontier RL pause from OpenAI, Anthropic declined to communicate a substantial parallel pause [1], although they discussed also briefly pausing some "high-risk" runs.
Anthropic has been the most vocal about AI risks, but it feels like all the big 3 have bought into the "others will do it if we don't do it first" narrative at this point. It increasingly gives "just following orders" vibes.
The HN majority and the VC crowd has been negligently complicit in downplaying AI safety, writing off Anthropic's statements as "hysteria" or "marketing", etc.
Now this capability will be coming to an open source model near you and every script kiddie will have a swarm of highly capable malicious agents. Now people care? Ridiculous.
You're telling me the DA would press charges without the actual harmed party wanting to press charges? I suppose it's possible but kinda hard without an actual plaintiff.
If they didn't conform to the T&Cs of the site (they almost certainly didn't), then they have violated the criminal law in some jurisdictions (e.g. Illinois criminalizes violations of T&Cs).
Per the linked article, "A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents"
German law for malicious computer use is fairly loosely defined, and massively favours the harmed party over the one causing harm.
I don’t think it’d be a slam-dunk by any means, but a reasonably competent legal team should be able to establish a case around malicious data interference at the least. There’s certainly enough merit to the idea that OpenAI would be better off settling it as a civil matter early.
This kind of incredulity in the AI era hilariously reminds me of the naivete of the late 90s. All of us edgy teenagers would be like "an MP3 is like just a long number, man! You can't own numbers!" Of course we were morons. In just the same way as the "oh they were just editing a wiki" defense is moronic.
The law isn't code. Human intent matters. Also when the really big number is a copyrighted song. Also when AI agents are set in motion to edit wikis or break in to websites.
>Huh? These are machines, built by their human builders. The humans are responsible.
i'm referring to the concept of intent, which at least when prosecuting under the US computer fraud and abuse act, is a critical component.
for example, creating a program that intentionally takes down a website is different than creating a program that has a bug which inadvertently takes down a website. in both cases, the person writing the code is responsible, but the consequences are different.
§ 823 Abs. 1 BGB, the general tort provision. It runs on negligence, so intent doesn't matter, and turning someone's wiki into an agent message board with 15,000 unwanted edits is a straightforward Verkehrssicherungspflicht breach.
Arguably, and not to offend you, it is those very people who have significant ownership stakes and funding in (Not)openAI.
It's not like the people with more resources than in any time in human history aren't investing in and wanting AI to succeed for their selfish reasons to grow their own resources and influence more. So, yes, it can "do whatever it wants" as long as most people remain weak, subservient, and disempowered to hold accountable those who keep making these decisions negatively shaping the majority's world.
you exclude yourself from the possibility space of people who can act and do anything becasue you resign. so be it- that's your choice. create that reality you desire.
I refuse that reality though, and I accept that a majority including I will unite. Good luck to you.
I mean, the article says that these were most likely "internal OpenAI agents" that were "internally deployed" and "clearly resemble a synthetic training or evaluation task." so yeah, OpenAI did this. Why they did this? Who knows? Maybe it was for testing, or marketing, but no one except OpenAI can say.
Great! They'll keep getting promoted until someone finds it disturbing and plausible enough to make another run for sama's house, like those 2 attempts in April!
"OH, we ALL of us need to be careful!" says OpenAI. No, you need to expect appropriate legals consequences for this sort of negligence -- you can't hide behind a GPU.