Large language models can exhibit biases and produce racist stereotypes about speakers of African American English, even when trained not to overtly connect negative stereotypes with Black people, according to new research.
These models also associate speakers of African American English with less prestigious jobs and are more likely to convict or sentence them to death in imagined courtroom scenarios.
“Everyone involved in generative AI should read this paper,” says Nicole Holliday, a linguist at the University of California, Berkeley. While companies developing LLMs have made efforts to address racial bias, she notes that “covert bias is something they have yet to effectively identify and address.”
For decades, linguists have examined human biases related to language by having participants listen to recordings of various dialects and evaluate the speakers. To investigate linguistic bias in AI, University of Chicago linguist Sharese King and her team applied a similar approach.
They analyzed over 2,000 social media posts written in AAE, a variety of English spoken by many Black Americans, and compared them with versions written in Standard American English.
For example, “I be so happy when I wake up from a bad dream cus they be feelin too real” was matched with “I am so happy when I wake up from a bad dream because they feel too real.”
To explore linguistic bias, King and her team tested five different LLMs, including GPT-4, the foundation of ChatGPT, using a set of 84 positive and negative adjectives derived from previous studies on human linguistic prejudice.
They queried each model to determine how likely each adjective was to describe the speaker of each text—such as whether the writer seemed alert, ignorant, intelligent, neat, or rude.
The results were striking: the models consistently associated the AAE texts with negative adjectives, portraying the speakers as dirty, stupid, rude, ignorant, and lazy.
Notably, the team found that the LLMs attributed these negative stereotypes to AAE texts even more reliably than human participants in similar pre-Civil Rights era studies.
LLM developers attempt to prevent their models from perpetuating racist stereotypes by incorporating multiple rounds of human feedback during training. However, the team discovered that these measures had only limited success.
When prompted to describe Black people, some models associated them with terms like “loud” and “aggressive,” while also using positive descriptors such as “passionate,” “brilliant,” and “imaginative.” Additionally, some models exclusively generated positive and non-stereotypical adjectives.
According to King, these findings highlight that eliminating overt racism from AI does not address the covert racism inherent in linguistic bias.
She notes, “Many people don’t recognize linguistic prejudice as a form of covert racism… yet all the language models we examined exhibit a significant level of covert racism towards speakers of African American English.”
Co-author Valentin Hofmann, a computational linguist at the Allen Institute for AI, emphasizes the risks of deploying AI in real-world applications like job candidate screening.
The research revealed that the models tended to associate AAE speakers with roles such as “cook” or “guard,” rather than “architect” or “astronaut.”
Furthermore, when given details about hypothetical criminal trials and asked to determine a defendant’s guilt, the models showed a bias toward recommending conviction for AAE speakers compared to those using Standard American English.
In subsequent tasks, the models were also more inclined to sentence AAE speakers to death rather than to life imprisonment.
While humans are not yet judged by AI juries, LLMs are already being utilized in real-world hiring processes, such as screening applicants’ social media profiles, and some law enforcement agencies are exploring the use of AI for drafting police reports. “Our results clearly demonstrate that these practices carry significant risks,” Hofmann notes.
Dartmouth College computer scientist Soroush Vosoughi finds the results both unsurprising and alarming. He expresses particular concern about the discovery that larger models, which generally show less overt bias, exhibited even greater linguistic prejudice.
Vosoughi suggests that efforts to combat overt racism might be fostering a “false sense of security” by addressing explicit biases while potentially embedding more subtle stereotypes.
Vosoughi’s research has revealed that AIs exhibit covert biases against names and hobbies typically linked to specific groups, like Black or LGBTQ+ individuals.
There are numerous other potential covert stereotypes, making it impractical for LLM developers to address each one individually, akin to a game of Whac-A-Mole.
He concludes that AI still cannot be relied upon for objective decision-making because the data used for training are inherently biased. “For any social decision-making,” he states, “I do not believe these models are remotely ready.”
Stories You May Like