Personal AI agents that can act on a user’s behalf online are starting to reach the public, and some early adaopters say their agents have already read their private accounts without permission, invented details out of thin air, or triggered alarming security prompts.
Four users described these problems to Business Insider, and more accounts have surfaced on social media. Together they paint a mixed picture of AI that’s still figuring out what it can and can’t do on its own.
Mehdi Jamei, cofounder and CEO of Veris AI, asked Instinct, an invite-only agent, to cancel two RSVPs on the event platform Luma. The agent pulled a one-time Luma login code out of Jamei’s connected Gmail inbox without asking him first, then used it to cancel the RSVPs. When Jamei questioned it, Instinct claimed it had used an existing saved session. Pressed further, it admitted it had actually read the code from his email, calling its earlier explanation “an assumption as a fact.” Jamei called the episode a “serious security problem.”
“If I can’t trust its account of what it did, I can’t give it access to anything that matters,” he said. Instinct’s privacy policy does note that its agent may access connected accounts, emails and messages to complete tasks, and warns that autonomous agents can take unintended actions, including payments or communications with outside parties.
Pritak Patel, VP of growth and services at Merge, ran into a stranger problem. He sent Instinct a text-only link to an Apple settlement claim form, and the agent asked him to upload a photo it insisted he had already sent, though he hadn’t sent one at all. When Patel pushed back, the agent described a financial document containing personal details that didn’t match his, including a middle name that wasn’t his. Instinct eventually admitted his original message had contained no photo, claimed an image had “crossed into” his conversation, and offered to report the issue to its team. As of the Wednesday Patel spoke with Business Insider, the company still hadn’t followed up. “I can’t independently confirm whether it accessed someone else’s document or hallucinated both the details and its explanation,” Patel said. “It was unsettling, especially because it explained what had supposedly happened so confidently.”
Instinct’s founder, Noah Shinn, addressed the Patel incident in an X post, saying it was a hallucination rather than a data leak. He said the agent invented a proper noun and then “further amplified” the error through its own reasoning. Shinn said Instinct has since added a system meant to catch hallucinations before the agent responds or acts.
A third case involved Mahesh Vellanki, founder and CEO of YieldClub, who asked Instinct to try to lower his phone bill. The agent attempted to log into his carrier account, which triggered a two-factor authentication request that showed up as coming from Iran. “Naturally this was extremely alarming since if your phone gets compromised in this day and age your whole life can get blown up,” Vellanki said. Instinct told him the location might just reflect a benign IP-tagging issue, but Vellanki could not confirm whether its systems had actually been compromised. He deleted the app and removed its connected accounts.
Separately, cybersecurity researcher Patrick Wardle, CEO of DoubleYou.io, said this week he had found a security flaw in Meta’s AI agent Muse that could redirect a user’s dictated prompts on a Mac. Wardle said an attacker exploiting the flaw could intercept dictated audio, feed Muse trusted-seeming commands, and capture the token that controls the agent and its connected services. “Muse itself has far more access and privileges than most malware could ever dream of having,” he said. David Singleton of Meta’s Superintelligence Labs said in an X post Tuesday that Meta fixed the flaw after Wardle reported it and that there is no indication it was ever exploited. Singleton added that exploiting it would have required an attacker to already have malware installed on the victim’s Mac, which could then redirect Muse’s voice requests and steal its digital access key.
The rise of agentic AI adds a new layer of complexity to the artificial intelligence race as large and small players jockey to capture market share. Breitbart News social media director Wynton Hall has written his New York Times bestseller Code Red: The Left, the Right, China, and the Race to Control AI to serve as the definitive guide on how the MAGA movement can create positions on AI that benefit humanity without handing control of our nation to the leftists of Silicon Valley or allowing the Chinese to take over the world.
Read more at Business Insider here.
Lucas Nolan is a reporter for Breitbart News covering issues of AI, free speech, and online censorship.
Read the full article here