A new study from Technion, Google Research, and Apple reveals that large language models, or LLMs, possess a deeper understanding of truthfulness than previously recognized. Traditionally, LLMs have been known to generate errors, referred to as “hallucinations,” which include factual inaccuracies and biases. The study analyzed the internal workings of LLMs by focusing on “exact answer tokens”—the specific response tokens that determine correctness—across four variants of Mistral 7B and Llama 2 models over ten datasets. Results indicated that truthfulness information is concentrated in these tokens, suggesting that LLMs encode their own truthfulness. The researchers developed “probing classifiers” that significantly improved error detection, demonstrating that LLMs have a multifaceted representation of truthfulness but do not generalize well across different tasks.
OpenAI’s AI transcription tool, Whisper, has been criticized for its excessive hallucinations. A University of Michigan researcher reported that eight out of ten audio transcriptions contained inaccuracies. An engineer analyzing 100 hours of transcriptions found hallucinations in about 50% of cases, while another developer noted issues in nearly all of the 26,000 transcripts he examined. Experts warn that this could disseminate misinformation across various industries, including healthcare, where Whisper is increasingly used to transcribe patient consultations.
This suggests that LLMs have a structured, internal way of encoding truthfulness through “exact answer tokens.” Focusing on these tokens could improve LLM reliability and potentially filter out hallucinations. For managed service providers (MSPs), this offers a new angle for advising clients on AI model selection, focusing on tools with embedded error-detection mechanisms for more accurate deployments in business-critical applications.
Businesses should be cautious about fully automating processes that rely on AI-generated outputs, especially those involving sensitive information. Implementing policies for review and verification, alongside robust logging to track AI-generated data, can help mitigate risks, and that’s all solution provider offerings.