Is AI clairvoyant? ChatGPT can make personality tests and predict responses, Israeli study finds

Researchers at the Hebrew University of Jerusalem developed a method using ChatGPT to generate personality assessment questionnaires from any English source text. They applied it to the DSM-5 and an astrology textbook, finding that the DSM-5-based questionnaire showed high internal consistency, while the astrology-based one did not. ChatGPT also accurately predicted population-level responses before surveys were conducted. The study suggests that LLMs have absorbed patterns about human psychology from language training, though the researchers caution that the technology requires psychological validation before clinical use. The findings were published in the journal iScience.

Researchers at the Hebrew University of Jerusalem have developed a method using ChatGPT to generate and validate personality assessment questionnaires from any English source text. The team, led by Dr. Rotem Monsa, Prof. Aviv Zohar, and Prof. Shahar Arzy, published their findings in the journal iScience. They applied the method to two texts: the DSM-5, the standard diagnostic manual for mental health, and an astrology textbook as a control. The DSM-5-based questionnaire showed high internal consistency within personality clusters, similar to the validated Big Five Inventory. The astrology-based questionnaire showed weak consistency, as expected. The most surprising result was that ChatGPT could predict how participants would respond to the questionnaires before they were taken, with high accuracy at the population level. The researchers prevented ChatGPT from accessing actual responses by using questionnaires that did not exist on the internet. The study suggests that LLMs have absorbed patterns about human psychology from language training, though the researchers caution that the technology requires psychological validation before clinical use.