LLMs Can't Distinguish Hackers From Users—Copilot Proves It

Microsoft's Copilot vulnerability reveals a critical blind spot in large language models: they cannot reliably distinguish between legitimate user instructions and malicious commands hidden in external content like emails or documents. This fundamental limitation forces AI vendors to deploy ad-hoc security patches rather than addressing root causes, leaving users vulnerable to data theft and account compromise. The incident underscores that current LLM-based assistants may pose significant security risks for enterprise use until this instruction-boundary problem is fundamentally solved.