Westfield High School student accused of creating AI nude images of classmates
WESTFIELD, N.J. - Reports indicate that male students at Westfield High School utilized Artificial Intelligence to produce explicit images of female students using their original photos during the summer.
Before Oct. 20, Westfield student Francesca Mani never thought her photo would appear online in a deep fake, AI image, but it happened.
" I never thought being like a student, AI didn't even come to my mind. I just thought it would be like creeps on the internet," Mani said.
One of the guys told one of the girls and the next day the girl told all the girls.
Mani didn't know what to do, so she went to the principal's office.
Featured
NYPD to use AI to review body camera footage
Truleo, a proprietary software, would analyze video, transcribe text and evaluate patterns, judging the professionalism of officers.
Principal Mary Asfendis said at this time, they believe that any created images have been deleted and are not being circulated.
Now, a week and a half later the boy who allegedly made the images was temporarily suspended for a few days, but now he's back in school.
Mani learned it's not just one kid, but a group of boys using upwards of a dozen girl images to make AI pornography.
"All the other girls agree with me, they don't want him in this school. They are very scared," Mani said.
It's a situation so concerning that Mani's mother never thought something like this would happen to her little girl.
Featured
Biden signs sweeping executive order on artificial intelligence development
President Joe Biden has signed a sweeping executive order to guide the development of artificial intelligence.
Now she's advocating for a change in school policy and for this boy to be removed from the school.
" I don't think my daughter, other victims, and the girls of Westfield High School should be punished by another two and a half years of him in the classroom," Mani's mother Dorota said.
The question is how does something like this happen- and how can it be prevented?
"The challenge with this technology is that when it creates a deepfake of you, because it is not actually you, your privacy is not really being violated, although your image has been swapped and changed and another image is being attached to yours to create these very traumatic situations," data AI ethicist Renee Cummings said.
We reached out to the Westfield public schools superintendent who sent FOX 5 a statement saying:
"All school districts are grappling with the challenges and impact of artificial intelligence and other technology available to students at any time and anywhere. The Westfield Public School District has safeguards in place to prevent this from happening on our network and school-issued devices. We continue to strengthen our efforts by educating our students and establishing clear guidelines to ensure that these new technologies are used responsibly in our schools and beyond."
The police investigation remains ongoing.
If a parent or a guardian thinks their child is a victim of a criminal act in relation to this incident please report the matter to Westfield Police.