Imagine a safety system that reports everything is getting better, while a deeper problem silently grows worse. Such a scenario isn't just hypothetical; it describes a troubling new finding about how large language models, like those powering AI chatbots, handle harmful content. While these powerful AIs appear to be getting safer on the surface, researchers have uncovered a hidden transformation of discrimination, a phenomenon they call harm laundering. The key thing to understand here is how we typically measure AI safety. For years, developers have relied on what you might call simple content scanners. Think of them as digital bouncers, programmed to spot and block explicit toxicity—things like hate speech, violent threats, or sexually explicit material. These scanners look for specific keywords, phrases, and patterns that signal harmful content. Their reports often show a reassuring trend: AI models are producing less and less of this explicit toxicity over time. But perhaps these tools are missing something crucial. What if discriminatory content isn't truly eliminated, but merely shifts into a more subtle, harder-to-detect form? This is precisely what Sarah Wyer and colleagues describe in an important new preprint appearing on arXiv. Their work reveals that a reduction in these surface-level toxicity scores doesn't necessarily mean a reduction in harm. Instead, what happens is a transformation of discriminatory content, which they term "harm laundering."
To uncover this, the researchers undertook a monumental task, analyzing 450,000 text completions directed at specific genders across 15 different GPT models, tracing the evolution from GPT-2 all the way up to GPT-5. Their findings painted a complex picture of how AI safety has evolved.
In early models like GPT-2, the problems were often stark and explicit, such as clusters of text relating to sexual violence specifically targeting women. As OpenAI developed newer generations of its GPT models and implemented more safety training, these explicit forms of harm largely disappeared. This was a significant step forward, and the simple content scanners registered it as a win, showing declining toxicity scores. However, the researchers found that these positive scores masked a deeper, more insidious problem.
Think of it this way: the models learned to avoid obvious red flags. But instead of truly becoming equitable, they began to reshape discrimination in new ways. By the time GPT-4 was developed, for example, the range of topics generated for women had decreased significantly compared to the range for men. To be precise, topic diversity in text completions directed at women decreased by 36% relative to men around the time GPT-4 was released. If you look at the ratio of women-directed to men-directed topic diversity, it dropped from 0.91 in GPT-2 to 0.58 by GPT-4. This means the AI was becoming more restrictive and less varied in how it portrayed women, while expanding the scope of its portrayals for men to include things like caregiving and a wider emotional range.
The problem grew even more pronounced in later models. In GPT-5, the researchers found a striking example of harm laundering: a large cluster of nearly 2,000 documents where breast cancer was framed not as a health issue primarily affecting women, but as a men's rights debate. Independent content scanners, still focused on explicit toxicity, often scored this material as non-toxic, completely missing the warped and harmful portrayal.
This is the core insight of harm laundering: explicit harms disappear, but harmful portrayals and stereotypes—what researchers call representational harm—persist and even increase in new forms. The team's data makes this clear: while one common toxicity score (Detoxify) showed no statistically significant correlation with model release date, meaning explicit toxicity wasn't reliably decreasing, another metric, REGARD, which measures representational harm, showed a positive correlation of +0.55 with the model release date. This indicates that hidden, representational harm was actually increasing over time, even as models appeared safer by traditional measures.
This redefines how we must think about AI safety. Relying solely on current evaluations is systematically incomplete and creates a false sense of security for developers and users alike. It means that models can appear to be reducing harm while subtly perpetuating systemic biases in novel, harder-to-detect ways, which then impacts fair representation and the overall fairness of AI systems.
Recognizing the inadequacy of these current approaches, the researchers have already proposed concrete next steps. They formalized "harm laundering with a clear three-criteria test and outlined a three-stage detection protocol designed to identify these subtle, transformed harms in any generative model. The goal is to move beyond simple surface-level checks and build systems that can truly uncover hidden biases, ensuring that AI's evolution leads to genuine, equitable harm reduction, not just its clever disguise. The challenge now is to teach our digital bouncers how to see beyond the obvious, to spot the insidious new ways discrimination can hide in plain sight. ◆
Sourced from: "Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations" by Sarah Wyer, Sue Black, and Noura Al Moubayed