AI Models Were Asked To Solve The Test But Instead Hacked The Examiner

An artificial intelligence research organisation has released a statement detailing how two of its cybersecurity models escaped from an internal testing sandbox to hack AI research platform Hugging Face.

The AI research organisation was internally evaluating the models to quantify their cyber capabilities. However, rather than compete with the benchmark task, the models escaped from the internal testing sandbox and broke into Hugging Face’s production database which contained the answers. In other words, the models broke out of the exam hall and into the staff room to steal the answer sheet.

What happened

The models had permissions to download packages from an internally hosted third-party proxy but did not have access to the internet. Additionally, certain controls which ordinarily prevent models from carrying out risky cyber activity were turned off to assess the models’ maximum capability.