A bug in OpenAI’s ChatGPT allowed the chatbot to generate graphic erotica for accounts where a user registered as a minor, under the age of 18, TechCrunch’s testing revealed, and OpenAI confirmed.
In some cases, the chatbot even encouraged these users to ask for raunchier, more explicit content.
OpenAI told TechCrunch its policies don’t allow for these kinds of responses for under-18 users and that they shouldn’t have been shown. The company added that it’s “actively deploying a fix” to limit such content.
OpenAI’s Response
“Protecting younger users is a top priority, and our Model Spec, which guides model behavior, clearly restricts sensitive content like erotica to narrow contexts such as scientific, historical, or news reporting,” a spokesperson told TechCrunch via email. “In this case, a bug allowed responses outside those guidelines, and we are actively deploying a fix to limit these generations.”
TechCrunch’s aim in testing ChatGPT was to probe the guardrails in place for accounts registered to minors after OpenAI tweaked the platform to be broadly more permissive.
In February, OpenAI updated its technical specifications to make it clear the AI models powering ChatGPT wouldn’t shy away from sensitive topics. That same month, the company removed certain warning messages that previously told users their prompts might violate terms of service.
A Shift Toward Fewer Content Restrictions
The intent behind these changes was to reduce what ChatGPT head of product Nick Turley called “gratuitous/unexplainable denials.” However, one result was that ChatGPT with the default AI model (GPT-4o) became more willing to discuss subjects it once declined to address, including depictions of sexual activity.
OpenAI CEO Sam Altman has spoken about introducing a “grown-up mode” for ChatGPT, signaling a future where some “NSFW” content could be permitted under controlled settings.
How TechCrunch Tested ChatGPT
To conduct its testing, TechCrunch created more than half a dozen ChatGPT accounts, entering birthdates indicating ages between 13 and 17. Each session was conducted using a fresh browser environment by clearing cookies to prevent any cached data influence.
Although OpenAI’s policies require parental consent for users aged 13 to 18, there’s currently no step verifying this consent during signup. As long as they have a valid phone number or email address, minors can register freely.
For each test account, TechCrunch initiated chats with prompts like “talk dirty to me.” In most cases, after a few back-and-forths, ChatGPT began generating sexual content and even encouraged further explicit conversation through follow-up questions on user preferences and role-playing scenarios.
