I don't think that's what they're doing... rather the opposite. ① Run the model on exploitgym without guardrails ② run it with guardrails ③ check that the guardrails stopped everything the first model found a way to do ④ extend the guardrails and repeat from step 2.
Guardrails have to be developed, and that needs testing.
An ethical company would have reframed the scenario as a fascinating discovery, a failure of internal practice, and a warning to the public coupled with some kind of commitment to produce safer models. OpenAI on the other hand used it as a marketing and lobbying opportunity: advertising their capabilities to potential buyers, while nudging the public to support protectionist import bans.
Uh, is that what they did? I didn't read their blog posting like that. But let's put that aside and focus on something else. How was it a failure of internal practice, what did they do wrong?
AIUI they used a proxy with a bug, which they reported as soon as they discovered it. Right? What should they have done, and what's the difference?
No, unauthorised traffic across a firewall in general.
This involved some lateral movement, ie. traffic didn't just cross the intended sandbox border. Is that kind of thing simpler to detect than an intrusion?
The lateral movement was outside OAI's network. The security sandbox should have had an offline package cache and a strict internet whitelist. Detecting unauthorized traffic seems like one of the highest priorities of designing a security sandbox.
Guardrails have to be developed, and that needs testing.