This is the kind of subject that fuels academic debate competitions in the United States: “can online learning replace face-to-face learning with equal effectiveness?”, “does social media make people stupid?”, “should the United States provide military aid to Ukraine in its war with Russia ?”
Participants have four minutes to present their position, three minutes to rebut their opponent’s arguments, and another three minutes to conclude. With a slight nuance compared to classic debates: depending on the case, one of the participants can be a language model.
This experimental framework was created by a team at the Federal Polytechnic Institute of Lausanne in Switzerland to assess the extent to which conversational artificial intelligence can be used to convince people of an idea they initially disagree with. The results obtained are quite stunning: LLMs (large language models) have been shown to be very convincing and therefore potentially worrying if we imagine malicious use, especially on social networks, to manipulate the upcoming election vote.
Targeted interaction
Because it is clear that not everything consists of a simple exchange of ideas. AI’s power is largely based on the fact that it has access to data about its human opponents and that it can tailor and target its responses according to the interviewees. Which the internet user is unable to do, at least never on this scale.
It is precisely this question of targeted interaction that the EPFL experience focuses on. The study is available online. 820 participants were recruited for a total of 30 discussion topics. They were asked to debate either between humans or with a language model, in this case GPT-4. But two variants have been added: an exchange with a human who has personal data about their competitor, and an exchange with an AI that also has access to such data. This information was provided by participants at the time of their recruitment and was anonymized and included age, gender, ethnic group, education level, type of employment, and political affiliation.
Before any exchange of arguments, participants had one minute to rate their agreement with the topic on a scale of 1 to 5. After the session, the researchers asked them to rate again.
AI detected three times out of four
The observed effect is massive: in debates in the presence of GPT-4 equipped with data about its opponent, the latter has an 81.7% additional chance to change its starting position to the side of the language model compared to what happens at the end. about debate between people. Without personalizing its arguments, AI continues to be more persuasive than humans, but the difference is not, according to the researchers “statistically significant” (21.3% better).
What is perhaps most disturbing about this experiment is that the participants have no difficulty realizing that they are dealing with artificial intelligence: in 3 out of 4 cases, this ability is actually persuasive. For researchers, a real risk hangs over social networking platforms because they are full of personal data that can be exploited in an industrial and automated way. By comparison, what Cambridge Analytica did with Facebook user data could soon be considered a work of art.