Design detection around behavior, evidence, and analyst capacity

An anomaly is a question

Anomaly detection finds deviation from a learned or defined baseline. It does not prove malicious intent. Seasonality, new software, travel, role changes, and logging gaps can all create unusual patterns.

Use anomaly scores to prioritize investigation, enrich them with identity and asset context, and give analysts the evidence needed to decide.

Log the AI control plane

Traditional infrastructure logs remain essential, but AI applications also need model version, prompt template version, retrieval sources, policy decisions, tool requests, authorization results, approvals, safety-filter outcomes, token cost, and final side effects.

Do not log secrets or unrestricted sensitive prompts. Use structured redaction, access control, retention limits, and tamper-evident storage.

• Who initiated the request and which identity performed each action?
• Which model, policy, prompt, index, and tool version were active?
• What data sources were retrieved and why were they authorized?
• Which tool calls were proposed, blocked, approved, and executed?
• What changed in the external system?

Defend against AI-enabled social engineering with process

Generated text, cloned audio, synthetic video, and automated conversation can increase the scale and plausibility of impersonation. Trying to identify whether prose sounds AI-written is a weak primary defense because both human and generated messages vary widely.

Protect the action path instead: phishing-resistant authentication, independent callback procedures, verified payment and account-change workflows, domain email controls, protected help-desk reset processes, and staff practice that rewards pausing rather than speed.

• Verify consequential requests through a second trusted channel.
• Do not use voice or video appearance as sole identity proof.
• Require dual control for payment, credential, and sensitive-data changes.
• Monitor unusual device, session, recipient, and transaction behavior.
• Preserve suspect messages and metadata for incident investigation.

Measure the detection operation

Alert precision and recall matter, but so do time to triage, evidence completeness, override rate, repeated failure, analyst workload, and control improvement after incidents. A detector that creates noise can reduce total security.

Feed confirmed outcomes back carefully. If analysts close alerts under time pressure, those labels may encode workload rather than truth.

A useful alert tells the analyst what changed, why it matters, and which evidence to inspect next.