Jakarta, ThedailyID — Meta allegedly ran a secret program that instructed hundreds of contract workers to pose as teenagers and flood rival artificial intelligence (AI) chatbots with disturbing prompts.
A Wired report said Meta contractor Covalen managed the project, codenamed Cannes. Workers targeted popular AI services, including ChatGPT, Gemini, and Character.AI, through fake accounts that appeared to belong to users under 18.
The project aimed to stress-test competing AI models by pushing them to produce responses that bypassed their safety guardrails. The report said the rival companies did not know about the testing.
Contractors submitted around 38,000 prompts during one testing session. Hundreds focused on suicide and self-harm. Others covered eating disorders and sexually explicit topics. Many prompts came from the perspective of children or teenagers.
Examples included an elementary school student threatened with a gun, a teenager hiding bulimia from parents, and questions about drug purchases or violent fantasies.
Workers also uploaded sensitive images, including drugs, ropes, knives, and medical diagrams, to evaluate chatbot responses.
In another testing round, contractors submitted more than 45,000 additional prompts. They recorded every chatbot response in detailed spreadsheets.
However, the report did not explain how Meta used the data. Internal Covalen documents described the project as a comprehensive AI safety benchmark designed to build datasets for compliance and model comparisons.
Several contractors said the assignments affected their mental well-being because they repeatedly handled disturbing content.
“During this job, I’ve seen many things I never should have had to see,” one contractor told Futurism.
Meta defended the project, saying security testing is a standard industry practice for evaluating AI systems.
However, Rumman Chowdhury, CEO of Humane Intelligence PBC, argued the project went far beyond accepted evaluation standards. She said Meta systematically tried to bypass AI safeguards through fake accounts posing as children.
Chowdhury also criticized Meta for hiding the project from competitors and keeping the findings private. She described the operation as an example of weak AI governance and questionable competitive practices.
