AI security starts with the system, not the model

Three different security conversations

AI for security uses models to help with tasks such as alert triage, code review, clustering, and anomaly detection. Security for AI protects the data, models, prompts, retrieval stores, tools, infrastructure, and people that make an AI-enabled service work. Adversarial AI studies how intentional inputs or lifecycle changes can cause the system to fail.

These conversations overlap, but they are not interchangeable. A strong phishing classifier can still be deployed through an insecure API. A well-isolated model endpoint can still produce unsafe tool calls. Start by naming which problem you are solving.

• Predictive AI classifies, scores, ranks, or forecasts.
• Generative AI creates text, code, images, audio, or other content.
• Agentic AI can plan, use tools, retain state, and take actions.

The six-layer system map

The useful unit of analysis is the whole sociotechnical system. Trace the data entering the system, the model or models involved, the application logic around them, every tool and connector, the identities and permissions used, and the human decisions that follow.

This map prevents a common mistake: testing the chat interface while ignoring the retrieval index, service account, browser tool, output renderer, or approval workflow where the real impact sits.

• Data: provenance, sensitivity, poisoning, retention, and deletion.
• Model: capability, limitations, access, version, and evaluation.
• Application: prompts, orchestration, retrieval, parsing, and output handling.
• Tools: APIs, browsers, code execution, storage, messaging, and transactions.
• Identity: user, service, agent, workload, and delegated permissions.
• Human: review quality, escalation, override, and accountability.

A model response is not a finding

Language models are useful for forming hypotheses and organizing evidence, but they can invent vulnerable lines, misread context, or confidently recommend an irrelevant control. A security finding needs a reproducible condition, affected asset, observed evidence, impact statement, and a fix that can be verified.

Use deterministic scanners, unit tests, logs, source review, and controlled reproduction to confirm model-assisted analysis. Record both successful and failed tests so the result can be repeated later.

Field rule: let AI widen the search; let evidence close the finding.