Unmoderated by Design: How Hugging Face Enables Non-Consensual Intimate Imagery
Unmoderated by Design: How Hugging Face Enables Non-Consensual Intimate Imagery
Unmoderated by Design: How Hugging Face Enables NCII
By investigating Hugging Face's Spaces, a platform designed for sharing open-source AI demos, we found a hub for tools capable of generating non-consensual intimate imagery, as well as for comparing, ...
Our investigation found that Hugging Face, the top platform for sharing open-source AI models, isn't just hosting tools that generate non-consensual intimate imagery. It's helping users compare, benchmark and optimize them, making image-based sexual abuse easier to scale.
By investigating the platform's most widely used Hugging Face Spaces and creating our own, we found:
- 73% of monitored requests were sexual in nature.
- Of those, 83% sought to undress or otherwise sexualize a person. 95% targeted women.
- 6.7% targeted a minor. If fulfilled, these requests would constitute child sexual abuse material (CSAM).
- Just 3% of audited Spaces showed any evidence of output moderation.
Hugging Face's own policies prohibit non-consensual intimate imagery. Yet we found virtually no safeguards preventing these tools from being used for exactly that purpose.