How Microsoft's AI security controls map to NIST AI RMF, ISO 42001, and the OWASP Agentic AI Top 10 — with gap analysis per clause. Includes upcoming regulatory deadlines for organisations deploying AI agents.
Microsoft's Zero Trust Workshop (microsoft.github.io/zerotrustassessment) is a free, open-source guided assessment framework built by the Microsoft Security CxE team. It provides pillar-specific assessment checks, a step-by-step deployment guide using a first-then-next structure, app permissions analysis, and workshop documentation. It is built from learnings across thousands of customer deployments. A formal AI pillar for the assessment tool is in development — expected summer 2026. Until then, architects should use the existing Identity, Data, and Networking pillar assessments alongside the new Zero Trust for AI reference architecture published at RSAC 2026.
Source: OWASP — LLM Top 10 for AI Applications (2025)
Distinct from the OWASP Agentic AI Top 10 below — the LLM Top 10 covers the full range of LLM application risks, while the Agentic Top 10 focuses specifically on multi-agent systems. Every AI agent you deploy is exposed to all ten. Use PyRIT to test for them before deployment.
| # | Risk | What it means for AI agents | Primary control |
|---|---|---|---|
| LLM01 | Prompt Injection | Malicious instructions override agent instructions. Includes XPIA (cross-prompt injection) via documents, emails, or web content the agent retrieves. | Prompt Shields, ATG, input validation |
| LLM02 | Sensitive Information Disclosure | Agent leaks PII, credentials, system prompts, or proprietary data in responses or via tool outputs. | Purview DLP, DSPM for AI, output filtering |
| LLM03 | Supply Chain | Compromised model weights, poisoned training data, malicious plugins, or unsafe third-party MCP servers. | Foundry model governance, MCP server vetting, Agent Governance Toolkit |
| LLM04 | Data and Model Poisoning | Adversarially modified training or fine-tuning data causes model to behave incorrectly or unsafely. | Foundry evaluation pipelines, model provenance tracking |
| LLM05 | Improper Output Handling | Agent outputs passed unsanitised to downstream systems — SQL injection via agent-generated queries, XSS via agent-generated HTML, command injection via agent-generated scripts. | Output validation, sandboxed execution, Foundry code execution controls |
| LLM06 | Excessive Agency | Agent has more permissions, tools, or autonomy than needed. Least agency principle violated. | Minimal connector/tool assignment, ATG tool allowlisting, least-privilege permissions |
| LLM07 | System Prompt Leakage | Agent reveals its system prompt or instructions — exposing business logic and enabling targeted attacks. | System prompt hardening, Prompt Shields, jailbreak detection |
| LLM08 | Vector and Embedding Weaknesses | Adversarial inputs manipulate RAG retrieval — poisoned documents inserted into knowledge base alter agent behaviour. | Document ingestion controls, retrieval validation, SAM RCD for SharePoint |
| LLM09 | Misinformation | Agent generates confident but incorrect information — dangerous in compliance, legal, medical, or financial workflows. | Human-in-the-loop for high-stakes decisions, Foundry evaluation, grounding with verified sources |
| LLM10 | Unbounded Consumption | Agent consumes excessive compute, tokens, or API calls — enabling denial of service or cost-based attacks. | Rate limiting, token budgets, ATG blocking, Azure AI throttling |
AI red teaming requires testing two surfaces simultaneously: security vulnerabilities (LLM01–LLM10) and responsible AI harms (bias, toxicity, manipulation). Traditional security testing focuses on only one. Microsoft's PyRIT automates testing across both surfaces — see the Products page for details and Playbooks for the pre-deployment workflow.
In December 2025, OWASP published the first formal taxonomy of risks specific to autonomous AI agents. Unlike the existing OWASP Top 10 for LLM applications (which focuses on model-level risks), the Agentic AI Top 10 covers risks that emerge when AI agents act autonomously — making decisions, invoking tools, and interacting with other agents. Microsoft's Agent Governance Toolkit (open source, April 2026) maps to all 10 risks.
| OWASP Risk | Description | Microsoft Control | AGT Coverage |
|---|---|---|---|
| Goal Hijacking | Adversary manipulates agent's objective through prompt injection or environmental data | Prompt Shield, Entra Internet Access Prompt Injection Protection | Semantic intent classifier in Agent OS policy engine |
| Tool Misuse | Agent invokes tools beyond intended scope — accessing unauthorised APIs, data, or systems | Foundry Guardrails, Defender for Cloud Apps CASB | Capability sandboxing + MCP security gateway |
| Identity Abuse | Agent impersonates users or other agents, acquires excessive permissions | Entra Agent ID, CA for Agents, ID Protection for Agents | DID-based identity + behavioural trust scoring |
| Supply Chain Risks | Compromised model, plugin, or dependency introduced into agent pipeline | Defender for Cloud AI model scanning, GitHub Advanced Security | Plugin signing with Ed25519 + manifest verification |
| Unsafe Code Execution | Agent executes unvalidated code or scripts with excessive privileges | Foundry execution sandboxing | Execution rings with resource limits |
| Memory Poisoning | Adversarial data injected into agent memory or RAG grounding data | DSPM for AI (grounding data blocking) | Cross-Model Verification Kernel (CMVK) with majority voting |
| Insecure Communications | Unencrypted or unauthenticated agent-to-agent communication | Entra Agent ID A2A protocol | Inter-Agent Trust Protocol (IATP) encryption |
| Cascading Failures | Failure or compromise in one agent propagates through multi-agent chain | Sentinel AI analytics rules, automated response rules | Circuit breakers + SLO enforcement |
| Human-Agent Trust Exploitation | Agent manipulates human oversight — bypassing approval workflows or creating false urgency | Human-in-the-loop controls in Copilot Studio | Approval workflows with quorum logic |
| Rogue Agents | Agent operates outside intended boundaries — ignoring instructions, self-replicating | Power Platform admin kill switch, Entra CA for Modern Agents | Ring isolation, trust decay, automated kill switch |
OWASP Top 10 for Agentic Applications 2026 · Microsoft Agent Governance Toolkit (GitHub, April 2026)
Two regulatory frameworks become enforceable in 2026 that directly apply to organisations deploying autonomous AI agents. These are not hypothetical — they have hard enforcement dates.
| Regulation | Enforcement Date | Who It Affects | Key Obligations for AI Agents |
|---|---|---|---|
| EU AI Act — High-Risk AI Obligations | August 2026 | Any organisation deploying AI systems classified as high-risk in the EU market | Risk management system, data governance, technical documentation, human oversight, accuracy and robustness requirements, logging and auditability obligations |
| Colorado AI Act | June 2026 | Developers and deployers of high-risk AI systems affecting Colorado consumers | Impact assessments, transparency disclosures, human oversight mechanisms, discrimination risk mitigation, consumer complaint process |
Most existing Copilot Studio agents are Classic agents — outside the Entra security perimeter with no lifecycle governance, no audit trail in Entra, and no automated kill switch. If your high-risk AI deployments include Classic agents, meeting EU AI Act auditability and human oversight obligations will require either migration to Modern agents or compensating controls. Microsoft's planned migration tool does not yet exist.
If you're standing up AI governance from zero, the first thing to run is the AI Baseline assessment in Purview Compliance Manager (Purview portal → Compliance Manager → Assessments → AI Baseline). It's a pre-built evaluation that automatically scores your tenant against the EU AI Act, NIST AI RMF 1.0, and ISO 42001 — surfacing remediation actions mapped to Purview, Entra, and Defender controls. Run it once to establish your baseline; re-run quarterly to track trend.
Beyond the AI Baseline, Compliance Manager includes additional AI-specific regulatory assessment templates that evaluate your tenant against specific obligations and surface prioritised improvement actions for data protection, auditability, and AI usage controls. It's the operational tool that turns each deadline into a task list. Access via the Microsoft Purview portal → Compliance Manager.
The Compliance Manager AI Baseline produces a posture score — useful for tracking trend and prioritising remediation. It is not the same as a structured compliance assessment with evidence collection, control testing, gap analysis, and a written findings report suitable for the ICO, EU AI Office, internal audit, or board sign-off. Regulated sectors (financial services, healthcare, public sector) typically need both: the score for operational tracking, and an independently validated assessment for regulator submission. Treating the score as the assessment is a common and significant misconception.
Most failed AI security programmes fail at governance, not technology. Compliance Manager produces evidence; Sentinel produces alerts; PyRIT produces findings. What turns those into sustained risk reduction is the human layer that meets to review them. The forums below are the minimum viable AI governance operating model — they sit alongside (not instead of) existing security governance.
| Forum | What it owns | Attendees | Frequency |
|---|---|---|---|
| AI Security Working Group | Cross-functional review of new agent deployments, the risk register, compliance posture, weekly KPI trends. Owns the agenda for everything below. | IT, Security, Data Protection, Legal, key business unit reps | Monthly |
| Agent Lifecycle Board | Approves new agents, reviews ownerless agents, owns the Classic-to-Modern migration roadmap, signs off on risk-tier overrides. Reviews every HIGH-tier agent. | Owner (per agent), Sponsor (per agent), IT Approver, security lead | Monthly |
| Quarterly Governance Sweep | Full Phase 1 KQL re-run, auth-type review, Access Package renewal, DLP exception review, ownerless-agent check cross-referenced with HR data, shadow-AI scan. | Security ops, IAM ops, Purview admin | Quarterly |
| Annual AI Risk Assessment | Full estate review against the risk tier rubric, red team prioritisation for the year ahead, compliance framework re-assessment, board pack preparation. | Working Group + executive sponsor | Annual |
| Agent Red Team Cycle | Structured adversarial testing of HIGH-tier agents, new agents tested pre-production, regression red teaming on significant change. Findings feed back into Agent Lifecycle Board. | Internal red team or external partner | Per new HIGH-tier agent + annual for in-production HIGH agents |
Working Group: direction and prioritisation. Lifecycle Board: approval and accountability for individual agents. Quarterly Sweep: operational hygiene. Annual Assessment: strategy and budget. Red Team: evidence. The forums escalate up the table — a Lifecycle Board cannot override the Working Group; an Annual Assessment cannot override the executive sponsor. Document the escalation path explicitly before the first meeting.
| Clause | Requirement | Microsoft Controls | Gap / Caveat |
|---|---|---|---|
| 4.2 — Interested Parties | Identify stakeholders and AI-related requirements | Agent 365 governance; Purview compliance; Entra Tenant Governance (preview) | Organisational process — not a product control |
| 5.2 — AI Policy | Establish and maintain an AI policy | SDL for AI; ZT for AI framework; Zero Trust Workshop (microsoft.github.io/zerotrustassessment) | Policy content is customer-defined; Microsoft provides scaffolding and guided workshop |
| 6.1 — Risk Assessment | AI-specific risk identification and assessment process | Security Dashboard for AI (now GA); Purview DSPM; AIAgentsInfo Advanced Hunting; Foundry Red Teaming | Quantitative risk scoring still limited; qualitative posture now available via GA dashboard. Classic Agent estate requires separate inventory. |
| 6.1.3 — AI Impact Assessment | Assess impacts on individuals and society | Microsoft Responsible AI Impact Assessment tools (separate from Security) | Outside security product scope; separate RAI tooling required |
| 8.4 — AI System Development | Security in AI development lifecycle | SDL for AI; GitHub Advanced Security; Foundry Red Teaming; Classic→Modern Agent migration | Classic Agent legacy complicates this — agents built before Agent ID may have no secure development baseline |
| 8.6 — Data for AI Systems | Data quality, provenance, and governance | Purview Information Protection; DSPM for AI; DLP for Copilot (GA March 31 2026) | Training data provenance still limited; inference-time data controls now stronger. Maker credentials can bypass data governance if not configured correctly. |
| 9.1 — Monitoring & Measurement | Continuous monitoring of AI system performance and risks | Security Dashboard (GA); Sentinel + MCP Entity Analyzer; Defender for AI; AIAgentsInfo KQL; Purview AI Observability | Good coverage when fully deployed. AI Agent Inventory requires Defender + Power Platform admin collaboration — complex setup. |
| 10.2 — Continual Improvement | Improve AIMS based on incidents and audit findings | Sentinel incident management; SDL feedback loops; ZT Workshop; ZT Assessment (AI pillar summer 2026) | ZT Assessment AI pillar not until summer 2026. Classic Agent name sync bug makes agent-level policy improvement tracking difficult. |
The GA of Security Dashboard for AI strengthens MAP and MEASURE function coverage. The discovery of the Classic vs Modern agent distinction reveals a gap across all four functions — most organisations cannot claim complete GOVERN, MAP, MEASURE, or MANAGE coverage until their Classic Agent estate is migrated to Modern Agents. This is the most significant framework compliance gap identified from field research and is not visible from Microsoft's product documentation alone.
Zero Trust isn't just for users and devices. The three core principles apply directly to AI agents, but the implementation looks very different from user-centric Zero Trust. Here's what each principle means in practice — and where the hardest gaps are today.
"Double agents" framing: Overprivileged, manipulated, or misaligned agents can act like double agents — working against the very outcomes they were built to support. This is Microsoft's framing for why standard least-privilege and assume-breach thinking must extend to AI agents, not just users.
Ephemerality Controls (JIT for agents): Agents should be granted short-lived credentials that expire the moment their specific task is completed. This Just-in-Time model limits blast radius if an agent is compromised mid-task — the attacker's access window is minutes, not days. Entra Agent ID supports this via time-bound access packages and lifecycle workflows.
Full AI lifecycle scope: ZT4AI covers not just agent runtime but the entire AI lifecycle — data ingestion, model training, deployment, and agent behavior. Supply chain and model security are in scope, not just the agent identity and access layer.
Alongside ZT4AI, Microsoft has introduced the Access Fabric concept — an architectural approach that treats access as a continuous, end-to-end system rather than a set of point controls. It uses identity as the consistent decision point and enforces those decisions across environments in near real time.
AI agents operate continuously, interact with multiple systems, and often require broad access. In a fragmented access environment, policy changes take longer to propagate, visibility is partial, and gaps between tools create openings. The Access Fabric model is directly relevant to Microsoft's agent security story — the same integrated Entra + Defender + Purview platform that Microsoft markets for agent governance is its implementation of this concept. The Classic vs Modern agent gap is a concrete example of what fragmentation looks like in practice: agents outside the Entra perimeter get zero coverage from the access fabric regardless of what other controls are deployed.
Don't try to implement all 700+ ZT Workshop AI controls (116 logical groups, 33 swim lanes) at once. This three-stage model gives organisations a practical sequence from zero visibility to full automation.
From the 700+ controls in the Microsoft Zero Trust Assessment Workshop AI section, these are the ones security architects should prioritise first.
From the 700+ controls in the Microsoft Zero Trust Assessment Workshop AI section, these are the ones security architects should prioritise first.
The 12 controls shown above are the highest-impact subset of the full Zero Trust Workshop AI catalogue. For the complete list of controls including effort, dependencies, and implementation notes, see the dedicated Zero Trust for AI page.