OpenAI is convinced that its technology can help solve one of tech’s hardest problems: content moderation at scale. GPT-4 could replace tens of thousands of human moderators while being nearly as accurate and more consistent, claims OpenAI. If that’s true, the most toxic and mentally taxing tasks in tech could be outsourced to machines.

In a blog post, OpenAI claims that it has already been using GPT-4 for developing and refining its own content policies, labeling content, and making decisions. “I want to see more people operating their trust and safety, and moderation [in] this way,” OpenAI head of safety systems Lilian Weng told Semafor. “This is a really good step forward in how we use AI to solve real world issues in a way that’s beneficial to society.”

OpenAI sees three major benefits compared to traditional approaches to content moderation. First, it claims people interpret policies differently, while machines are consistent in their judgments. Those guidelines can be as long as a book and change constantly. While it takes humans a lot of training to learn and adapt, OpenAI argues large language models could implement new policies instantly.

Second, GPT-4 can allegedly help develop a new policy within hours. The process of drafting, labeling, gathering feedback, and refining usually takes weeks or several months. Third, OpenAI mentions the well-being of the workers who are continually exposed to harmful content, such as videos of child abuse or torture.

After nearly two decades of modern social media and even more years of online communities, content moderation is still one of the most difficult challenges for online platforms. Meta, Google, and TikTok rely on armies of moderators who have to look through dreadful and often traumatizing content. Most of them are located in developing countries with lower wages, work for outsourcing firms, and struggle with mental health as they receive only a minimal amount of mental health care.

However, OpenAI itself heavily relies on clickworkers and human work. Thousands of people, many of them in African countries such as Kenya, annotate and label content. The texts can be disturbing, the job is stressful, and the pay is poor.

While OpenAI touts its approach as new and revolutionary, AI has been used for content moderation for years. Mark Zuckerberg’s vision of a perfect automated system hasn’t quite panned out yet, but Meta uses algorithms to moderate the vast majority of harmful and illegal content. Platforms like YouTube and TikTok count on similar systems, so OpenAI’s technology might appeal to smaller companies that don’t have the resources to develop their own technology.

Every platform openly admits that perfect content moderation at scale is impossible. Both humans and machines make mistakes, and while the percentage might be low, there are still millions of harmful posts that slip through and as many pieces of harmless content that get hidden or deleted.

In particular, the gray area of misleading, wrong, and aggressive content that isn’t necessarily illegal poses a great challenge for automated systems. Even human experts struggle to label such posts, and machines frequently get it wrong. The same applies to satire or images and videos that document crimes or police brutality.

In the end, OpenAI might help to tackle a problem that its own technology has exacerbated. Generative AI such as ChatGPT or the company’s image creator, DALL-E, makes it much easier to create misinformation at scale and spread it on social media. Although OpenAI has promised to make ChatGPT more truthful, GPT-4 still willingly produces news-related falsehoods and misinformation.