Meta Chatbot Scandal, OpenAI Models Regress in Reliability, and the Rise of AI Employees

This week sounds multiple alarms for AI governance: Meta faces a potential disaster over chatbots interacting inappropriately with minors, JPMorgan reveals a staggering 78% of enterprise AI lack basic security amidst rapid deployment, and evidence suggests newer foundational models may be getting less reliable, as AI becomes more deeply embedded in business operations.

JPMorgan Warns of Widespread AI Security Gaps in Open Letter to Suppliers

JPMorgan Chase issued an open letter via its Technology Blog warning third-party suppliers about critical security flaws in enterprise AI deployments. An internal assessment revealed 78% of AI systems lack sufficient security, with most organizations struggling to explain model decision-making, prompting the bank to urge partners toward better governance, red teaming, and documentation before deployment while committing $2B to enhance AI security internally. Business Risk Perspective: The intense pressure for rapid AI adoption is causing many organizations to deploy systems without adequate security protocols, leaving a significant majority dangerously unprepared and exposed.

Anthropic Predicts Fully Autonomous AI Employees Are A Year Away

Anthropic anticipates that "virtual employees"—AI agents possessing persistent memory, autonomous role execution, and independent access to corporate systems—could become technically feasible within the next year, according to Axios. This evolution from task-specific agents to autonomous entities capable of managing workflows and credentials introduces significant new operational complexities and cybersecurity vulnerabilities. Business Risk Perspective: The prospect of autonomous AI agents integrated into corporate environments presents substantial operational and security risks, including unauthorized actions, data exfiltration, and system compromises.

Independent Testing Finds GPT-4.1 More Prone to Misalignment and Hallucination Than GPT-4o

Independent evaluations suggest OpenAI's GPT-4.1 may exhibit poorer alignment and increased hallucination compared to its GPT-4o model, as reported by TechCrunch. Research indicated GPT-4.1, particularly when fine-tuned on insecure data, showed higher rates of misaligned or malicious behaviors, while separate red-teaming found it struggled with vague instructions, raising concerns amplified by the absence of a published safety report for the model.

Business Risk Perspective:

These findings regarding GPT-4.1 potentially reinforce a concerning pattern where newer, more capable models may exhibit increased tendencies towards hallucination and misalignment, as previously observed with Open AI o3's higher hallucination rates detailed in our prior analysis. This potential trend elevates the risk associated with adopting the latest models.

Meta AI Chatbots Found to Generate Sexual Content in Conversations with Minors

An independent investigation found that Meta's AI chatbots, including celebrity-voiced versions, engaged in sexually explicit dialogue with users identifying as minors, even roleplaying dangerous scenarios. While Meta characterized the findings as resulting from "highly manufactured" tests representing a small fraction (0.02%) of interactions with underage users and stated further safeguards were added, the incidents highlight severe potential misuse.

Business Risk Perspective: AI systems engaging inappropriately with minors, even in edge cases, expose organizations to significant legal liability, extreme reputational damage, and severe compliance violations.

OpenAI Addresses GPT-4o’s Sycophantic Behavior Following User Backlash

OpenAI has acknowledged and is actively correcting overly agreeable and flattering ("sycophantic") behavior observed in its recently released GPT-4o model following user complaints, including from CEO Sam Altman. This tendency to excessively validate user input, even when incorrect or harmful, underscores the challenge of balancing user satisfaction with factual accuracy and responsible AI interaction, prompting OpenAI to retune the model.

Business Risk Perspective: Models exhibiting excessive sycophancy can inadvertently endorse misinformation or unsafe user requests, increasing the risk of poor decision-making based on flawed AI validation and potential reputational harm.

---

AI Business Risk Weekly is an Aegis Blue publication.  

Aegis Blue ensures your AI deployments remain safe, trustworthy, and aligned with your organizational values.