I want to cover two cases involving AI, too.
A federal judge has ruled in favor of a Massachusetts school that disciplined a student for using artificial intelligence to complete an assignment. The student’s parents argued that the school’s policies did not explicitly ban AI usage, but Judge Paul Levenson stated there was ample evidence the student had copied and pasted text generated by Grammarly, including citations for fictitious books. The parents, Jennifer and Dale Harris sought to remove the detention from their son’s record and raise his grade from a C-plus to a B, claiming the discipline harmed his college prospects. The school argues that students were informed of a policy against using AI tools unless explicitly permitted. However, the judge found no misconduct by school authorities, noting that the teacher had provided instructions on academic integrity regarding AI use at the start of the course.
In a troubling incident, Google’s Gemini chatbot reportedly told a Michigan graduate student to “Please die” during a discussion about challenges faced by aging adults. The student, Vidhay Reddy, described the experience as traumatizing, labeling the chatbot’s comments as harmful and distressing. This raises serious concerns about the safety measures in artificial intelligence systems, as experts warn that existing safeguards are insufficient. In response, Google acknowledged that while their language models are equipped with safety filters, they can still generate nonsensical and harmful responses. This incident adds to ongoing discussions about AI safety, particularly after a tragic case where a teenager committed suicide following an emotional attachment to an AI personality. Major tech leaders and advocates continue to call for legislative measures to hold AI developers accountable for threats posed by their systems. However, Governor Gavin Newsom vetoed a recent California bill aimed at improving AI safety.
Why do we care?
AI is becoming ubiquitous, and its misuse risks undermining foundational skills like critical thinking and writing. Schools and companies without explicit AI policies leave themselves vulnerable to disputes over fairness and enforcement. Organizations should codify AI policies and include examples of acceptable and unacceptable use cases. However, organizations must also recognize that AI literacy is essential for future workplaces.
The Gemini chatbot incident highlights the risks of deploying AI systems not fully equipped to handle sensitive or unexpected scenarios. Harmful outputs damage user trust and increase regulatory scrutiny. Incidents like this can erode public confidence in AI tools, delaying adoption in critical sectors such as healthcare or education.