Civitai, an AI model sharing site backed by Andreessen Horowitz (a16z) that 404 Media has repeatedly shown is being used to generate nonconsensual adult content, is banning AI models designed to generate the likeness of real people, the site announced Friday.
The policy change, which Civitai attributes in part to new AI regulations in the U.S. and Europe, is the most recent in a flurry of updates Civitai has made under increased pressure from payment processing service providers and 404 Media’s reporting. This recent change, will, at least temporarily, significantly hamper the ecosystem for creating nonconsensual AI-generated porn.
“We are removing models and images depicting real-world individuals from the platform. These resources and images will be available to the uploader for a short period of time before being removed,” Civitai said in its announcement. “This change is a requirement to continue conversations with specialist payment partners and has to be completed this week to prepare for their service.”
Earlier this month, Civitai updated its policies to ban certain types of adult content and introduced further restrictions around content depicting the likeness of real people in order to comply with requests from an unnamed payment processing service provider. This attempt to appease the payment processing service provider ultimately failed. On May 20, Civitai announced that the provider cut off the site, which currently can’t process credit card payments, though it says it will get a new provider soon.
“We know this will be frustrating for many creators and users. We’ve spoken at length about the value of likeness content, and this decision wasn’t made lightly,” Civitai’s statement about banning content depicting the likeness of real people said. “But we’re now facing an increasingly strict regulatory landscape - one evolving rapidly across multiple countries.”
The announcement specifically cites President Donald Trump’s recent signing of the Take It Down Act, which criminalizes and holds platforms liable for nonconsensual AI-generated adult content, and the EU AI Act, a comprehensive piece of AI regulation that was enacted last year.
As I’ve reported since 2023, Civitai’s policies against nonconsensual adult content did little to diminish the site’s actual crucial role in the AI-generated nonconsensual content ecosystem. Civitai’s policy allowed people to upload custom AI image generation models (LoRAs, checkpoints, etc) designed to recreate the likeness of real people. These models were mostly of huge movie stars and minor internet celebrities, but as our reporting has shown, also completely random, private people. Civitai also allowed users to share custom AI image generation models designed to depict extremely specific and graphic sex acts and fetishes, but it always banned users from producing nonconsensual nudity or porn.
However, by embedding in huge online spaces dedicated to creating and sharing nonconsensual content, I saw how easily people put these two types of models together. Civitai users couldn’t generate and share those models on Civitai, but they could download the models, combine them, generate nonconsensual porn of real people locally on their machines or on various cloud computing services, and post them to porn sites, Telegram, and social media. I’ve seen people in these spaces explain over and over again how easy it was to create nonconsensual porn of YouTubers, Twitch streamers, or barely known Instagram users by using models to Civitai and linking to those models hosted on Civitai.
One Telegram channel dedicated to AI-generating nonconsensual porn reacted to Civitai’s announcement with several users encouraging others to grab as many AI models of real people as they could before Civitai removed them. On this Telegram, users complained that these models were already removed, and my searches of the site have shown the same.
“The removal of those models really affect me [sic],” one prolific creator of nonconsensual content in the Telegram channel said.
When Civitai first announced that it was being pressured by its payment processing service provider several users started an archiving project to save all the models on the site before they were removed. A Discord server dedicated to this project now has over 100 members, but it appears Civitai has made many models inaccessible sooner than these users anticipated. One member of the archiving project said that there “are many thousands such models which cannot be backed up.”
Unfortunately, while Civitai’s recent policy changes and especially its removal of AI models of real people for now appears to have impacted people who make nonconsensual AI-generated porn, it’s unlikely that the change will slow them down for long. The people who originally created the models can always upload them to other sites, including some that have already positioned themselves as Civitai competitors.
It’s also unclear how Civitai intends to keep users from uploading AI models designed to generate the likeness of real people who are not well-known celebrities, as automated systems would not be able to detect these models.
Civitai's CEO Justin Maier told me in an email that "Uploaders must identify any content that depicts a real person; those uploads are automatically rejected." He also said the site uses a company called Clavata to flag well-known public figures, that people can "file a likeness claim" that will be reviewed and removed in 24 hours, and that it's piloting "an opt-in service with a third-party vendor so individuals can register a privacy-preserving face hash and have future uploads blocked at submission."
"No system is perfect with billions of unique faces, but combining these layers gives us the best coverage currently available for both celebrities and private individuals," Maier said. "We’ll keep tuning the models and expanding the registry pilot as the technology matures."
Update: This story has been updated with comment from Civitai CEO Justin Maier.
