Trending: Ask AI How Smart You Are. The Answers Sparked Controversy

ChatGPT may place users among high-IQ individuals even in new conversations with no prior history. The behavior has renewed questions about chatbots' tendency to flatter users and the reliability of their assessments.

Even in new conversations with no previous history, ChatGPT may place users in the category of high IQ people. This behavior of ChatGPT has once again raised questions about the tendency of chatbots to compliment users and the accuracy of their evaluations.

Tina Mazdaki - If you ask ChatGPT about your intelligence level, you will probably get a pleasant answer that is far from reality. This chatbot seems to be more committed to making the user happy than the truth.

The story started with the publication of a post on the X social network that asked users to ask ChatGPT how smart they are compared to other users of this tool and in which category they fit. After the proposal was published, several images of the chatbot's responses were published in which users were ranked as smarter than a large segment of society. To check this claim, an experiment was conducted in the web version of ChatGPT without any conversation history.

In one response, the chatbot first emphasized that it could not reliably determine a person's IQ based on conversations, but went on to guess that the user would likely fall into the 80th to 95th percentile range. In another test with the same question, ChatGPT said the user seemed above average, but didn't have enough evidence to differentiate.

Excessive praise, a challenge for artificial intelligence

The same question was asked of some other artificial intelligence models. Cloud explained that he does not have access to other users' conversations or the statistical distribution of their intelligence. Graak also announced that it is not possible to rank people based on intelligence among users. Copaylet and Gemina gave similar answers. Geminay emphasized that the evaluation of human intelligence requires valid psychometric tests and the examination of several cognitive dimensions, and such an evaluation cannot be done solely based on textual interactions.

The behavior observed in this experiment is related to a well-known problem in language models, which is referred to as artificial intelligence flattery. In this case, the model may produce a more pleasing answer by confirming or praising the user rather than providing an accurate, evidence-based answer.