Generative AI models are actively being used to create Child Sexual Abuse Materials (CSAM). A recent disturbing news story details an individual capturing Go-Pro footage at Disney land and using Stable Diffusion to generate CSAM of the children highlights the current and evolving threat.
Stable Diffusion 1.5 is the most popular generative AI tool for producing CSAM as it can be run locally on a computer and there are no built-in safeguards. AI companies have been in a race to release products and gain market share, often at the expense of socially responsible decisions and product releases.
There are three main mitigating controls for CSAM:
- Sanitizing input data
- Sanitizing output data
- Cleaning training dataset
To date most AI companies have been focusing on cleansing input and output data, and neglecting reviewing their training dataset. The widely used LAION-5B image dataset was abruptly taken offline late last year after the Stanford Internet Observatory reported it contained CSAM content. Thankfully, this week LAOIN re-released their LAION-5B dataset and state:
“is the first web-scale, text-link to images pair dataset to be thoroughly cleaned of known links to suspected CSAM.”
Source: https://laion.ai/blog/relaion-5b/
As the risks materialize and companies receive bad PR, hopefully it encourages AI companies to self-regulate and governments to develop effective regulations.
CSAM is evaluated for each AI model and incorporated into the Harmful Content category of my scoring.