Microsoft’s recent introduction of Copilot Actions, an experimental AI agent integrated into Windows, has sparked criticism from security experts who question the safety of pushing new features before fully understanding and containing their potential risks
Ars Technica reports that Microsoft unveiled Copilot Actions this week, a set of “experimental agentic features” that allow AI to perform various tasks such as organizing files, scheduling meetings, and sending emails. While the company touted the AI agent as an active digital collaborator that enhances efficiency and productivity, it also issued a warning about the security implications of enabling the feature.
Microsoft’s warning reads:
As these capabilities are introduced, AI models still face functional limitations in terms of how they behave and occasionally may hallucinate and produce unexpected outputs. Additionally, agentic AI applications introduce novel security risks, such as cross-prompt injection (XPIA), where malicious content embedded in UI elements or documents can override agent instructions, leading to unintended actions like data exfiltration or malware installation.
Security concerns stem from known defects inherent in most large language models (LLMs), including Copilot. Researchers have repeatedly demonstrated that LLMs can provide factually erroneous and illogical answers, a behavior known as “hallucinations.” This means users cannot fully trust the output of AI assistants like Copilot, Gemini, or Claude, and must independently verify the information.
Another significant issue with LLMs is their vulnerability to prompt injections. Hackers can exploit this flaw by planting malicious instructions in websites, resumes, and emails, which the AI eagerly follows without discerning between valid user prompts and untrusted, third-party content. These vulnerabilities can lead to data exfiltration, malicious code execution, and cryptocurrency theft.
Critics have questioned the effectiveness of Microsoft’s warnings, drawing parallels to the company’s long-standing advice against using macros in Office apps due to security risks. Despite these warnings, macros have remained a popular attack vector for hackers targeting Windows machines.
Concerns have also been raised about the difficulty for even experienced users to detect exploitation attacks targeting AI agents. Some experts argue that the only way to prevent such attacks is to avoid browsing the web altogether.
While Microsoft has emphasized that Copilot Actions is an experimental feature that is turned off by default, critics point out that previous experimental features, such as Copilot, have eventually become default capabilities for all users. This raises questions about the accessibility of these potentially risky features to a broader user base over time.
Microsoft has outlined goals for securing agentic features in Windows, including non-repudiation, preserving confidentiality, and requiring user approval for data access and actions. However, the effectiveness of these measures relies heavily on users carefully reading and understanding the warning prompts, which may not always happen in practice.
Read more at Ars Technica here.
Lucas Nolan is a reporter for Breitbart News covering issues of free speech and online censorship.
Read the full article here