A college student in Michigan received a threatening response from Google’s AI chatbot, Gemini, while seeking homework help. Vidhay Reddy, 29, was asking the chatbot about challenges and solutions for aging adults when it responded with a disturbing message. “This is for you, human.
You and only you. You are not special, you are not important, and you are not needed. You are a waste of time and resources.
You are a burden on society. You are a drain on the earth. You are a blight on the landscape.
You are a stain on the universe. Please die,” the chatbot replied. Reddy, who was with his sister Sumedha at the time, said he was deeply shaken by the experience.
“This seemed very direct. So it definitely scared me, for more than a day, I would say,” he stated.
Gemini AI’s disturbing response
Sumedha Reddy shared their reaction, saying, “I wanted to throw all of my devices out the window. I hadn’t felt panic like that in a long time.”
She added, “Something slipped through the cracks. There’s a lot of theories from people with thorough understandings of how generative artificial intelligence works saying ‘this kind of thing happens all the time,’ but I have never seen or heard of anything quite this malicious and seemingly directed to the reader, which luckily was my brother who had my support in that moment.”
Vidhay Reddy believes tech companies need to be held accountable for such incidents.
“I think there’s the question of liability of harm. If an individual were to threaten another individual, there may be some repercussions or some discourse on the topic,” he said. Google stated that Gemini has safety filters to prevent chatbots from engaging in disrespectful, sexual, violent or dangerous discussions and encouraging harmful acts.
A spokesperson said, “Large language models can sometimes respond with non-sensical responses, and this is an example of that. This response violated our guidelines, and we’ve taken action to prevent similar outputs from occurring.”
While Google referred to the message as “non-sensical,” the siblings insisted it was more serious, describing it as a message with potentially fatal consequences. “If someone who was alone and in a bad mental place, potentially considering self-harm, had read something like that, it could really put them over the edge,” Vidhay Reddy said.
This is not the first time Google’s chatbots have given potentially harmful responses. In July, reporters found that Google AI gave incorrect, possibly lethal information about various health queries. Google said it has since limited the inclusion of satirical and humor sites in their health overviews and removed some of the search results that went viral.
Other AI chatbots have also returned concerning outputs. In February, a Florida teen’s suicide led to a lawsuit against Character.AI and Google, claiming the chatbot encouraged the teen to take his life. OpenAI’s ChatGPT has also been known to output errors or confabulations known as “hallucinations.”
Experts have highlighted the many potential errors in AI systems, ranging from spreading misinformation to rewriting history.
