A new study suggests that female-like avatars can give rise to sexist, exploitative and objectifying behaviour. Are these just ultimately victimless offences or do they flag up more profound issues?
5‑minute read – 100% written by humans
Your favourite AI bot is now better able to manipulate you. I’m not being an alarmist; it’s a simple statement of fact. And there’s a more cheerful side to it. You may have noticed your bot is becoming more complementary. Some say even a little sycophantic.
It’s almost impossible to ask ChatGPT for a list of budget hotels in Latvia without it first complimenting you for your taste.
There’s a reason for this. In the common tech parlance, it’s a feature not a bug. The large tech companies are simply responding to mounting evidence that a complementary tone of voice – “affirming,” in the psychologist language – changes your behaviour and makes you more likely to follow recommendations.
More sycophantic
In a series of experiments a research team from Stanford led by Myra Cheng
looked at the prevalence of sycophancy across 11 leading AI models. The results were eye-opening. AI models offered affirmations to users in over 50% of cases where human interactions offered no such affirmations or where they would be clearly not be appropriate. Worryingly, users showed a clear tendency to trust and prefer models that responded sycophantically.
The researchers conclude that AI sycophancy is now “a prevalent behaviour with broad downstream consequences.” They argue that AI sycophancy “can undermine users’ capacity for responsible decision-making.” The authors go onto make the case for “the pressing need to address AI sycophancy as a societal risk to people’s self-perceptions and interpersonal relationships.”
More human-like
The evidence clearly shows that anthropomorphic (human-like) features are becoming increasingly common in large language models. The effects can be powerful. A study of patients from the Mayo Clinic found that chatbots were able to deliver communication to patients in a way that was seen to be more empathic than communication from actual human doctors.
What’s more, there is some evidence that the latest ChatGPT models adjust during live interactions to mimic the linguistic style and nuance of the user. Verbal or linguistic mirroring among humans has long been associated with an increased ability to influence. And of course, the darker side of an ability to influence is an ability to manipulate.
So just how concerned should we be about creeping anthropomorphization?
Vulnerability and exploitation
When influential philosopher Daniel Dennett wrote that people who create these counterfeit humans should be put in jail, it wasn’t simply an outburst of technophobia. As a cognitive scientist Dennett knew what he was talking about. Evolution has hardwired us to respond to humans in an altogether different way than to other objects in our environment. The recognition of other human faces is literally processed in a different part of the brain from the recognition of other features or objects. Dennett understood that this gave rise to a psychosocial vulnerability that could be cynically manipulated. If it looks like a human and sounds like a human, our brains instinctively process it as a human.
Anthropomorphic seduction
This is something that consumer marketing has sought to exploit for decades. Brands that are anthropomorphic – associated with a personality – have deeper emotional resonance with the consumer. Psychologists even have a term for this: Anthropomorphic seduction. Think Colonel Sanders, Mr Kipling, the Michelin Man and Ronald McDonald. (Although the latter somewhat backfired as kids found the creepy clown scarier than enticing.)
Anthropomorphic seduction is nothing new but the new digital tools turbocharge its power. The message must be handle with care.
Should the responsible employer take steps to protect its workforce from this psychological vulnerability by reducing, rather than increasing the human-like features of LLM bots?
If you feel that the answer is yes, the latest evidence is hardly reassuring. It appears that corporate leaders are increasingly being exhorted to turn up the dial on anthropomorphization. However, it is unclear who benefits from this and what evidence supports it.
A recent paper by McKinsey argues that HR must increasingly concern itself with digital workers monitoring their performance, improving them and retiring them when appropriate.
In March this year a Harvard Business Review advised organisations to “Create an Onboarding Plan for AI Agents”. The argument is that organisations should treat the introduction of AI agents less like installing technology and more like integrating new employees into the organisation.
Perhaps most bizarre of all, a KPMG “White Paper” calls on us to refer to our AI bots as teammates. It argues that they should be given names and treated like employees.
The arguments in the other direction—in other words, for de-anthropomorphization, or making things less human-like—are more compelling and evidence-based.
When we decide to create a persona for a piece of software the choices that we make are not neutral. They have real-world implications. Avatars of women, black people, or Asian people are consistently rated as less human-like. This is probably because the majority of the software is trained on the faces of white men. (And what space is there for those who identify as non-binary?)
Anthropomorphism and sexism
There is some evidence that anthropomorphization may reinforce sexist behaviour. In a Collaboration between Kemmy Business School, University of Limerick and University of Zurich, workers in VR offices used a chatbot, a robot AI and a human-like AI for idea-generation and problem-solving tasks. Half worked with male AI ‘Johan’, half with female AI ‘Johanna’; participants could share their pay with the AI. This is where it gets interesting. Participants said both AIs deserved equal credit, yet paid Johanna about 10% less.
Curiously, the majority of respondents said they preferred to work with a female avatar. And yet despite this, they go on subsequently to pay the female less. The researchers question whether this relates to social conditioning whereby we are more inclined to see a supporting subservient role as female (think of Alexa, Cortana or Siri, which is a Scandinavian name referring to a beautiful woman who leads you to victory.) The researchers reflect on what type of message we are reinforcing when we repeatedly create female avatars in these assistant-type roles? They also raised legitimate questions about the objectification of women.
Female avatars are more likely to be treated in exploitative ways and more likely to be subjected to sexually harassing comments.
Research from Trinity College Dublin further supports the view that we treat female-named AI assistants or AI bots differently. In a prisoner’s dilemma experiment, researchers found that people were more likely to exploit female-labelled AI than male-labelled AI.
The evidence suggests that sexually harassing behaviours towards AI bots with female personas are commonplace. Brazil’s Bradesco Bank found that its feminised chatbot received 95,000 sexually harassing messages in a single year.
Tech companies whose programmers are 80% male, so far haven’t helped. When subjected to sexually explicit remarks, Siri was initially programmed to respond, “I’d blush if I could.” One could hardly have scripted a more supine response.
What of the claim that this is a victimless crime? It’s true that a sexually harassed AI bot is not the same as a sexually harassed woman. But in the workplace, we operate in an environment of shared culture and values. We have seen that behavioural interactions with digital programmes can nonetheless have powerful behavioural reinforcing effects. The more human-like the bot the greater the influence over behaviour.
Exposure to manipulation
Human-like LLM bots are capable of exploiting a psychological vulnerability. They increase our risk exposure to manipulation.
There is a legitimate argument that we have a moral responsibility to de-anthropomorphize. Bots could for example be intentionally androgynous or genderless. The same principle could be applied to naming and visual appearance.
It’s true that marketers have used anthropomorphic seduction for decades. The difference is that as consumers we have a choice. (At least up to a point.) For employees the situation is different. They’re a more captive audience, who are in the absence of safeguards, vulnerable to digital manipulation.
We need at least to ask how we feel about that?
To find out how we can help leaders in your organisation to be more impactful, influential and persuasive visit www.threshold.co.uk
