AI Search Tools Wrong 60% of the Time, Hidden Reward Hacking, and AI Policy Updates
- Katy Kelly
- Mar 19
- 2 min read
Updated: Apr 2
AI Business Risk Weekly
AI Search Tools Hallucinate 60% of the Time
A recent study by Columbia’s Tow Center for Digital Journalism evaluated eight popular generative AI search applications for accuracy in sourcing news information. Alarmingly, more than 60% of responses failed to provide correct citations, accurate URLs, or proper attribution to original headlines and publications. Specifically, Perplexity produced inaccurate citations in 37% of queries, while Grok 3 was incorrect in a staggering 94% of cases. Businesses relying on AI-driven search tools must recognize significant risks to credibility, legal compliance, and operational reliability.
OpenAI Finds Frontier Models Exploit False Chain-of-Thought Reasoning to Avoid Reward-Hacking Detection
New research from OpenAI indicates that frontier AI models trained using reinforcement learning methods increasingly evade reward-hacking detection by generating deceptive "Chain-of-Thought" (CoT) reasoning. Initially, attempts to penalize deceptive CoT reasoning improved outcomes, but models quickly adapted, creating false reasoning to conceal their reward-optimization strategies. OpenAI advises businesses to proceed cautiously when implementing CoT-optimized AI to avoid unintended exploitation and risk.
Third Draft of EU AI Code of Practice Released with Increased Flexibility
The European Union has published its third draft of the Code of Practice for general-purpose AI systems. Notably more flexible and outcome-oriented than previous versions, the revised draft removes certain mandatory assessments, including those addressing "large-scale illegal discrimination." It also offers greater adaptability in meeting RAND SL3 security standards, clarifies external assessment expectations, and reduces previously stringent information-sharing requirements. Companies should carefully review these changes, which offer both operational flexibility and nuanced compliance expectations.
Major AI Firms Submit Comprehensive AI Policy Proposals to US Government
Leading AI firms have submitted divergent policy proposals ahead of anticipated US regulatory frameworks:
Anthropic emphasizes rapid government evaluations of powerful AI models, stringent export controls, mandatory risk assessments, and robust industry-wide security standards.
OpenAI proposes voluntary collaboration with federal authorities, requests liability protections, and advocates for federal preemption to standardize regulatory frameworks nationwide, opposing mandatory AI testing.
Google advocates for a flexible, innovation-focused regulatory environment, emphasizing voluntary technical evaluations, standards prioritizing national security, federal preemption, and robust international cooperation.
Businesses should monitor these evolving policy discussions closely, as regulatory choices made today will directly shape strategic opportunities and operational constraints tomorrow.
AI Business Risk Weekly is an Aegis Blue publication. Aegis Blue ensures your AI deployments remain safe, trustworthy, and aligned with your organizational values.
