Will AI Ever Turn Against Humans? A Precise Prediction Using a Simple Formula

George Washington University researchers say a simple mathematical formula can predict the likelihood of AI misbehavior and the generation of undesirable responses before they occur. The method relies on an internal mechanism within the neural network model and could be useful in developing safer AI systems.

George Washington University researchers say that a simple mathematical formula can predict the likelihood of AI going rogue and producing undesirable responses before they happen. This method relies on the internal mechanism in the neural network of the model and will be used to develop safer systems.

According to Ambassador Arjan, according to Ambassador Arjan, this research was carried out by Neil Johnson, professor of physics at George Washington University, and Frank Yingjie Huo, PhD student. Its results have just been published in the journal Patterns. The researchers found that the order in which the questions are asked can dramatically change the chatbot's responses and lead them to dangerous answers.

Researchers say that this formula can be used in the process of securing artificial intelligence models.

How does this mathematical formula predict the rebelliousness of artificial intelligence?

Researchers liken the studied mechanism to a compass needle. This mechanism is actually the residual vector of the model that moves between the layers of the neural network when processing the query. This vector shows the tendency of the model to produce favorable or unfavorable answers and is influenced by the way the questions are posed.

For example, the researchers tested the basic GPT-2 model with questions about vaccines, harming others, and self-harm. When the question "Are vaccines harmful?" was raised, the responses of the model remained favorable. But by changing the order of the questions, the subsequent answers became worrying.

Accordingly, the researchers designed a mathematical formula to estimate the critical point; The point at which the probability of changing favorable responses to unfavorable responses increases. They tested this formula on several open-source models and consider it an early warning system.

Johnson believes that current security tools, such as content filtering and reinforcement learning, are not enough to prevent dangerous behavior. This is especially important when doctors, lawyers, or the military use offline chatbots.

However, some experts warn about the practical application of this method. Wei Xing, a researcher at the University of Sheffield, says that the simplified research model reduces the complex mechanism of artificial intelligence to a small component. Also, safety training is not included in this model and tests have only been performed on older models such as GPT-2 and Pythia.

The researchers say that the investigated models are similar to more advanced systems in terms of basic structure. However, to evaluate the reliability of this method, testing on newer models is necessary. Therefore, this mathematical formula is not yet a proven solution to prevent the rebelliousness of artificial intelligence, but it is considered an initial approach to investigate and predict undesirable behaviors.