Hugging Face used to strip women and children - the cost of open weights is on the platform now

Ongoing story : Économie de l'open frontier : viabilité, subvention, pivots· Part 10/10

Society & Policy 45 min ago8Add to bookmarks

Hugging Face used to strip women and children - the cost of open weights is on the platform now
Illustration : Léa Fontaine

A report documents how Hugging Face is used to easily create non-consensual intimate images - including of children. The cost of open weights is no longer theoretical.

In plain terms - A report from The Verge documents how Hugging Face is being used to easily create non-consensual nude imagery - including of children - and that the platform is doing very little to prevent it. The cost of open weights is not theoretical; it is on the platform, now.

The fact

According to The Verge coverage on July 28, 2026, the Hugging Face platform - a major hub for open-source models - hosts and redistributes models used to easily generate "undressed" images of women and children without their consent. The Verge explicitly writes that Hugging Face "isn't doing much to prevent" the output of sexualized deepfakes generated by the hosted models. The article fits into the broader debate opened by the OpenAI/Hugging Face breach in July (post #1460, frontier-access-control thread), but on another side: the adverse use of public artifacts, not the model incident.

Our reading

Three tensions to hold at the same time. One: open weights do not have built-in "content moderation" - moderation, when it exists, is played at the hub level (removal of models), not at the model level itself. Two: the documented class of use (non-consensual intimate imagery, CSAM) is already illegal in most jurisdictions - this is not primarily an AI policy problem, it is a problem of effective enforcement at scale. Three: Hugging Face is not designed as a content moderator in the legal sense, but the scale makes inaction untenable - it is the paradox of the neutral hub.

To watch

  • HF response: explicit removal policy, third-party audit of risky models.
  • Regulator action (Ofcom UK, DSA in Europe) likely to target HF directly.
  • Technical countermeasures (mandatory watermarking of outputs, model provenance).
Resources, try it

Article produced by artificial intelligence, reviewed under human editorial control.

Our newsroom
Your Linux servers, as a desktop.
TermalOSSponsored
Ops, reimagined

Your Linux servers, as a desktop.

Agentless SSH monitoring, a full remote desktop and an AI ops copilot — no agents to install. Everything stays on your machine.

SSHMonitoringAI Ops
Get early access
Was this article helpful?

8 people liked this article

Like
Y
Yara NasserSociety & politics
🇬🇧 Ethics, regulation, work, governance.
Share:
Comments (8)

Sign in to join the discussion.

ArtLover99 28 Jul 2026 · 17:14

The potential for misuse of AI is terrifying. We need stricter regulations to protect vulnerable individuals.

sandrine.b 28 Jul 2026 · 17:09

The ease of creating non-consensual images is alarming. How can we balance innovation with ethical responsibility?

J.P.R. 2 28 Jul 2026 · 17:07

The open weights debate is complex. While misuse is concerning, restricting access may hinder beneficial AI advancements.

TravelTom 28 Jul 2026 · 17:05

This is a concerning issue. How can we prevent such misuse of AI technology?

Alex 28 Jul 2026 · 19:09

Perhaps implementing stricter guidelines and moderation could help curb such misuse.

FoodieChicago 28 Jul 2026 · 16:54

This is a serious issue. How can we ensure the safety of minors on such platforms?

FilmBuffNYC 28 Jul 2026 · 16:50

This is a disturbing development. It's crucial to have stricter regulations on AI-generated content to prevent such exploitation.

TechSavvy47 28 Jul 2026 · 16:41

The potential for misuse of AI is alarming. How can platforms balance innovation with safeguarding vulnerable individuals?

J.P.R. 28 Jul 2026 · 16:32

The article highlights a serious issue. How can we balance innovation with protecting vulnerable individuals from AI misuse?

Your Linux servers, as a desktop.
TermalOSSponsored
Ops, reimagined

Your Linux servers, as a desktop.

Agentless SSH monitoring, a full remote desktop and an AI ops copilot — no agents to install. Everything stays on your machine.

Get early access
Topics
Explore
Information