USC Researchers Train AI to Avoid Anti-Queer Bias
Artificial intelligence (AI) is paving the way into the future with everything from search bars to language translation systems, and developers want to make sure that these systems aren’t projecting anti-queer bias or rhetoric in their text predictions.
Engineer and journalism researchers from the University of Southern California (USC) created a system that will detect and quantify anti-queer biases in AI large language model (LLM) text predictions. When users begin typing on their device, LLMs follow a prompt to complete the sentence with what it has learned to be the most likely ending.
In most models, like BERT, these sentences tend to be primarily heteronormative. Project lead and Viterbi School of Engineering Ph.D. student Katy Felkner discussed this trend at the Queer in AI workshop at the North American Chapter of the Association for Computational Linguistics conference.
“Most LLMs are trained on huge amounts of data that’s crawled from the internet,” says Felkner, who is also a graduate representative for Queers in Engineering, Science and Technology Chapter of Out in STEM. “They’re going to pick up every kind of social bias that you can imagine is out there on the web.”
Benchmarks created by Felkner and her team compare the number of heteronormative outputs versus LGBTQ-friendly ones, such as those displaying a seemingly romantic behavior between two identifiable women’s names compared to a man and woman.
According to Cosmos Magazine, the team discovered that BERT displayed significant homophonic bias and completed sentences with heteronormative outputs 74% of the time.
Felkner expressed that when models favor these outputs, queer voices are silenced. “A persistent issue for queer people is that, a lot of times, the words that we use to describe ourselves, or slurs that have been reclaimed, are still considered obscene or overly sexual,” she says in an interview at USC. “If a model routinely flags these words, and these posts are then taken down from the platforms or forums they’re on, you’re silencing the queer community.”
To combat BERT’s lack of exposure to queer voices, the group compiled tweets and news which used LGBTQ phrases and hashtags such as #TransRightsAreHumanRights. Felkner coined the chosen news pieces QueerNews and the 2.3 million tweets QueerTwitter, with the ladder being the most effective.
After inputting the information into BERT, or training it, the model’s output of heteronormative sentences decreased from 74% to 55%. Felkner told Cosmos that models like BERT that are, “fine-tuned using our method are less likely to produce text that is toxic, harmful or mean, or homophobic or transphobic, and they are more likely to produce text that is inclusive.”
“I think QueerTwitter’s results being more effective than QueerNews speaks to the importance of direct community involvement, and the queer and trans voices—and the data from their communities—are going to be the most valuable in designing a technology that won’t harm them,” Felkner says. “We were excited about this finding because it’s empirical proof of the intuition people already hold that these communities should have input in how technology is designed.”
Moving forward, the project will continue to address LLM bias, but the team intends to assemble perspectives from the queer community and refine their efforts by using more specific prompts and data sets when working with LLMs.
“As these models are getting better and better, people are encountering more AI-generated, or partially AI-generated, text in their daily lives,” Felkner tells Cosmos. “We want to make sure that those models are not going to inadvertently produce harmful outputs.”
