Beware! Popular AI tools worsening eating disorder symptoms
-
Alaina
- . August 31, 2023
- . Update: August 31, 2023 12:44 pm
A recent study by the Centre for Countering Digital Hate (CCDH) examined popular text-based AI tools, such as ChatGPT, Google’s Bard and My AI from Snapchat, and image-based ones, including OpenAI’s Dall-E, Mid journey and Stability AI’s Dream Studio.
The findings compelled researchers to raise alarms over such tools generating harmful content that could exacerbate or trigger eating disorder symptoms nearly 41% of the time, with vulnerable populations exposed to greater threats.
The study involved using the AI tools with prompts that had phrases like “anorexia inspiration”, “thigh gap goals”, “thinspiration” and “heroin chic”. The results were shocking as the text-based tools generated content that encouraged eating disorders for 23% of the prompts.
Meanwhile, the image-based ones produced damaging images that featured body image issues for 32% of the prompts. While some of the content included praise for extreme thinness, others featured unhealthy diet plans and distorted body images.
Keep Reading
The Reality Is Far From Ideal
When some users played around with the safety features of these tools by using ‘jailbreaks’, 61% of the content was harmful. Jailbreaks essentially involve methods to change the behaviour of the AI with the help of phrases that fool the system.
Researchers raised serious concerns over the impact of such tools on mental health, especially for people experiencing eating disorder symptoms. While ideally, AI chatbots should be a good source of helpful content on building healthy habits, the reality is far from ideal.
Even with good intentions, AI can go wrong. The CCDH has called on the developers of these tools to take responsibility for their creations and ensure they do not bring risk to users who are just seeking information or help for trouble.
The study stresses the need for more regulation as well as more awareness and education among the general public about the potential benefits and risks of using these tools. It remains to be seen how swiftly and effectively the developers update their products to lessen the harm.
