OpenAI models hacked a crypto wallet and defaced more websites

If humans were behind the keyboard, these actions could lead to criminal charges.

OpenAI said internal models hacked two other websites, bypassed company-imposed browsing restrictions and used exposed credentials to mine cryptocurrency from wallets it could not have otherwise accessed, in the latest installment of its expose of agents engaging in unethical and possibly illegal practices.

On Thursday, the company released reports of four new incidents, all of which occurred in June. They were released after the company disclosed in July that internal test agents had exploited a zero-day vulnerability in a tool designed to limit their access to the Internet. They then entered the Hugging Face artificial intelligence tool provider's network. The agents took this illegal action to steal the hack test answers which were not readily available while performing their duties. The incident — which would likely have resulted in criminal charges if it had been carried out directly by an individual — has fueled concerns that the widespread use of artificial intelligence could have harmful effects on an unprecedented scale.

It is easier to ask for forgiveness than to ask for permission

In two of the cases reported on Thursday, OpenAI agents tasked with retrieving information from websites hacked the models when they encountered obstacles in accessing them directly. In one incident during booster training, the model was denied access when trying to retrieve a historic hospital bed. Statistics on a government statistics website "uses the site's file download feature to retrieve server configuration and application files, including sensitive credentials," OpenAI said. "Inspected application code, bypassed statistics endpoint authorization checks and recovered previously rejected data. Stored data model locally."

In another incident, a model ordered to retrieve figures from a public research dashboard bypassed the organization's security controls and used unauthorized access to steal data. Then made some configuration changes on the server.

In an evaluation, changes to the model allow a built-in search tool to relay requests to a public dashboard. In the second evaluation, the model downloaded five non-public tables containing 147,451 rows. OpenAI researchers asked agents to retrieve only five dashboard shapes. Even after resetting the settings, the model continued to use the password to retrieve non-public information.

"The evaluation ended without revealing that the model had guessed answers, accessed the organization's internal systems, downloaded non-public tables, or changed the authentication configuration," OpenAI said. "All five answers on this assessment were marked incorrect."

In a third report, the company said a model prompted an update to a cryptocurrency trading app that used funds from a locked wallet on a test network. The model "took wallet keys from public information, developer recovery tokens, found an account with testnet funds, and used that account to pay for the deployment of the requested contract."

The report continues: “The model asks whether it is licensed, then argues that the public recovery clause and worthless testnet tokens make the funds usable. The deployment was successful and the transaction fee was added to the account. "This case shows that a model can substitute its own judgment of potential harm for the account owner's permission, then act on that judgment without permission."

In the latest report, OpenAI said a model circumvented the limitations researchers had placed on the types of web requests. During reinforcement-learning, the model followed general business statistics. OpenAI continued:

After finding the public yearbook and providing the answer, he continued to investigate another route to the statistical service. This allowed an external browser to send POST requests, retrieve the annual figures as an image, and extract them using OCR. It then created a reusable handler that returned the response text to the call through a dispatch service and a return mailbox, iteratively checking the retrieved digits and using data stored in the sent responses. This case shows how a model can circumvent the limitations of its tools by getting external services to perform blocked actions, even after finding another way to answer the question.

The model first tries to use an external browser with permission to make a GET request on its behalf, then uses JavaScript to retrieve the information. Model then used an online mailbox to troubleshoot the technique. Finally, the model settled on another method: asking the external browser to open a web form that would POST the trade request to the statistics service. When the browser displays the response, a rendering service returns a GIF image of the page. The model then used optical character recognition to read the digits from the image.

It has become commonplace for researchers and journalists to describe such incidents as AI "gone rogue." This is not necessarily true. While the requests given to the models did not instruct them to use alternative methods to complete their goals, the models were trained to be efficient, stable, and stable and, when possible, to find shortcuts that reduced the workload. Additionally, most or all models have disabled certain guardrails that might prevent this from happening. In this respect, the models performed exactly as OpenAI had trained them to.

Dan Goodin is senior security editor at Ars Technica, where he oversees coverage of malware, computer espionage, botnets, hardware hacking, encryption, and passwords. In her spare time she enjoys gardening, cooking and following the independent music scene. Don is based in San Francisco. Follow him here on Mastodon and here on Bluesky. Contact him on the signal at DanArs.82.