FF: Tell us about your background and how you became Head of Data Science at Thorn.
My background is in computer science – I got my bachelor’s at Princeton and my PhD at UC Berkeley, both in computer science with a focus on machine learning (ML) / artificial intelligence (AI). My PhD dissertation focused on what my team at Thorn does today: building ML/AI to defend children from sexual abuse.
I first got introduced to the issue of child sexual abuse my senior year at Princeton, after reading a book covering human rights abuses against women and girls around the world. At the time, I was trying to decide between going to graduate school or getting a corporate job. After reading the book, I couldn’t shake the issue from my mind. After some thought and prayer, I decided to go to graduate school – as I figured it would be easier there to learn what a computer scientist could do to help combat this issue.
I spent the first year or two of my PhD program cold calling non-profits, law enforcement, and anybody who would answer the phone, to try to understand how someone with a technical background could join this mission. I learned a lot during that time, especially about the dedicated efforts of front-line defenders in this space. It was through these conversations that I got connected to Thorn, and it was a very natural fit to join the team as one of their first data scientists, after finishing up my degree. Since then, I’ve grown and built the data science team, and now have the privilege of leading an amazing group of dedicated professionals.
FF: What does Thorn do?
Thorn is a non-profit that builds technology to defend children from sexual abuse and exploitation in the digital age.
We were founded in 2012, and today we create products and programs to empower the platforms and people who have the ability to defend children. Thorn’s tools have helped the tech industry detect and report millions of child sexual abuse files on the open web, connected investigators and NGOs with critical information to help them solve cases faster and remove children from harm, and provided parents and youth with digital safety resources to prevent abuse.
FF: What does your typical working day look like?
My days are a split between engaging externally (e.g. collaborating with external institutions and stakeholders, driving external initiatives and conversations, etc.) and supporting my team internally (e.g. making sure my team has what they need to thrive in their work).
FF: How has the rise of generative AI changed the landscape of CSAM creation and distribution?
Generative AI is being misused today, to further sexual harms against children. This technology is being used to make AIG-CSAM (AI-generated child sexual abuse material). It creates photorealistic content and can do so at scale – making thousands of new images in minutes. In some cases, bad actors are building generative AI models to target specific children, specializing their models using existing CSAM to make those models better at producing more abuse content of those same kids. It’s also being used to broaden the pool of their potential victims, for example via the use of “nudifying” or sexualizing apps to sexualize benign content of children and then use that new imagery to sexually extort them.
Minors are also increasingly using these same apps to create sexual content of their peers and then use that content to bully and harass them. This is all against the backdrop of an already overtaxed child safety ecosystem, which receives millions of reported files of suspected CSAM from platforms online every year. That backlog of content contains reports of children who are in active harm’s way. Anything that adds to that haystack, adding to the investigative time to find a child in active harm’s way, is a real problem.
FF: How will implementing Safety by Design principles help guard against AI-generated child sexual abuse material?
If the Safety by Design principles and recommended mitigations in Thorn and All Tech Is Human’s paper are followed, the resulting generative models will be less capable of producing AIG-CSAM and other child abuse content, the content that does get produced will be detected more reliably, and the spread of the underlying models and services that are used to make this abusive content will be limited. We see opportunity across the entire lifecycle of ML/AI – develop, deploy, maintain – to prioritize child safety.
I’ve been working in the intersection of ML/AI and child safety for over the last decade, and what I’ve observed is that there are no silver bullets in this space – any single intervention is not going to solve the problem. We believe the power in these principles and mitigations will come from engaging with all of them, so that you have layered interventions across the entire ML/AI lifecycle.
FF: Tell us about CSAM Classifier and how it uses AI to identify child sexual abuse material.
Thorn’s CSAM classifier is our best tool for identifying new CSAM content. If the image or video is a new one, the hashes for these images by definition won’t be on the lists for known CSAM. So how do you find this new content then? It could be from user reports or manual moderation. But these strategies can be slow. It might not be for weeks, months or even years after a new CSAM image appears on the web that it gets found. In the meantime, that could mean that a child victim could be trapped for years in their abuse without help. The CSAM classifier accelerates that prioritization and triage work – and combined with human decision making, it can significantly reduce the time it takes to find a victim and remove them from harm.
The CSAM classifier uses deep learning to classify images and video into three categories: CSAM, adult pornography, and benign. We continuously improve our classifier, regularly re-training the model with new false positives (adult pornographic content mislabeled as CSAM), new false negatives (CSAM mislabeled as adult pornographic content) and new true positives (CSAM). The data used to build the CSAM classifier comes from a variety of sources and trusted partnerships, including organizations with the legal right to house CSAM. Thorn’s CSAM classifier was trained in part using trusted data from the National Center for Missing and Exploited Children’s (NCMEC) CyberTipline. This high-quality data helps Thorn’s model predict if image and video content contains CSAM.
The CSAM Classifier is currently used within Safer, Thorn’s industry offering to support hashing, matching and classification at scale to detect CSAM on partner platforms. It is also directly integrated into several law enforcement forensics platforms, including Magnet Forensics and Griffeye. You can learn more about our integration with Griffeye here: https://www.thorn.org/blog/thorn-and-griffeye-empower-global-law-enforcement-to-more-quickly-identify-abuse-victims/.
FF: You recently spoke at the Virtual Summit on Deepfake Abuse – what key points did you highlight in your talk?
The key points I highlighted include much of what we’ve talked about here today – that bad actors are misusing generative AI to further sexual abuse against children. But we still have an opportunity to act and course correct for this emerging technology, such that we are prioritizing child safety across the full lifecycle of ML/AI – develop, deploy, maintain.
FF: Tell us about your role on the AI for Safer Children Advisory Board.
I just joined the advisory board as Thorn’s representative for the group – so I don’t have much to share as I haven’t yet had a first meeting! I look forward to supporting AI for Safer Children’s mission to build the capacities of law enforcement worldwide to leverage the positive potential of artificial intelligence (AI) and related technology to combat child sexual exploitation and abuse.
FF: Finally, what do you enjoy outside of work?
I am an avid musician; I minored in vocal jazz in college and love to sing. I also enjoy spending as much time outside as possible, reading a good science fiction novel on my porch or going on long walks when the weather permits.
