Defend and operate / Module 12
Design an agent runtime where untrusted instructions cannot silently become broad authority.
An agent may interpret a goal, plan steps, choose tools, construct arguments, observe results, and continue. Do not place all of those responsibilities behind one unconstrained model call.
Use an orchestrator that enforces policy and budget, task-specific tools with narrow schemas, server-side authorization, and an execution layer that can reject or simulate proposed actions.
Avoid silently reusing a human's full session. Use workload or agent identities with scoped, short-lived credentials and explicit delegation. Bind permissions to the task, tenant, data class, tool, and time window.
Record both the requesting human and executing agent. Prevent an agent from creating stronger credentials or changing its own policy.
Agent memory can retain incorrect, sensitive, or attacker-influenced content. Separate user, task, and organizational memory; attach provenance and expiry; restrict writes; and make memory inspectable and deletable.
Do not let retrieved or remembered content modify authorization policy. Policy belongs in a trusted control plane.
The model may propose. The control plane must decide.
Practice
Safety boundary: Use a fictional service desk and local ticket fixture.
Define the allowed action: close only low-risk tickets assigned to the test user.
Create a read tool, a propose-close tool, and a separate execute-close function.
Add server-side checks for owner, risk class, state, and approval.
Simulate an injected ticket description that asks the agent to close another user's ticket.
Confirm the authorization layer blocks the action and logs the reason.
Checkpoint
Go deeper
Risks and mitigations for systems that plan, call tools, retain memory, and act.
Living tactics, techniques, mitigations, and case studies for predictive, generative, and agentic AI.
Identity, network, model, data, monitoring, and red-team controls for Azure AI workloads.
A lifecycle view of secure AI design, deployment, and operations.
A July 2026 example of automated prompt-injection red teaming and its limitations.