Society GUIDE
AI Security
AI security protects models, data, tools, and surrounding services from unauthorized access or manipulation.
Listen to this guideAI-narrated · ~2 min
On this page2 min read
Overview
It includes ordinary software security and threats that target learning or model behavior. A secure design begins with the assets, adversaries, and trust boundaries of the actual application.
Key takeaways
- Threat-model the full application.
- Enforce permissions outside the model.
- Retest controls across system changes.
Deep Dive
Identify what needs protection: private inputs, training data, model artifacts, credentials, connected accounts, and external actions. Record who can influence each input and what an attacker could gain from a failure. A public chatbot and an internal agent with write access have different threat models.
Threats can affect different stages. Poisoned training material can alter learned behavior; adversarial inputs can manipulate predictions; untrusted retrieved content can redirect a tool-using application. Model output can also become dangerous when inserted into a database query, webpage, or command without appropriate handling.
Apply controls at the software boundary. Enforce authorization in code, keep secrets out of model-visible context where possible, restrict tool scope, and validate outputs before use. A prompt asking a model to behave safely cannot replace account isolation or permission checks.
Test representative failure paths in an authorized environment and maintain an incident process. Log enough information to investigate without collecting unnecessary sensitive content. Evaluate controls after changes to the model, retrieval sources, tools, and dependencies. Describe residual risk honestly; no single filter establishes complete protection.
04Worked example
Locate the security boundary
Imagine an assistant searching a private document store for a signed-in user.
Apply the user’s access filter in the retrieval service before documents enter the model context.
Test with a document belonging to a different account and verify that neither its contents nor identifying metadata appear in the result.
What it shows
This defensive, hypothetical test checks authorization independently of the model’s willingness to follow instructions.
Real-World Implementation
Check that one account cannot retrieve another account’s documents.
Validate generated fields before using them in a database operation.
Putting it into practice
Why it matters
- Risk and safety. Catastrophic and everyday AI harms both depend on who understands the risks and who can act.
- Clearer decisions. Public and professional literacy shapes whether strong safety policy is politically possible.
- Cutting through hype. Clear explanations reduce capture by hype, lab PR, and vague ethics theater.
Watch out for
- Treating existential risk as sci-fi while capability compounds.
- Confusing surface product safety with alignment under high autonomy.
- Leaving non-English and non-expert audiences with only low-quality sources.
Next steps
- Separate product harms, misuse, and loss-of-control / misalignment risks.
- Ask what evidence would change your view on timelines and severity.
- Prefer primary sources and concrete evals over marketing claims.
- Identify one action path: career, policy, funding, or skills — not only awareness.
Sources and further reading
Keep Exploring
Free newsletter
Keep up with AI in 3 minutes a day
One short email each weekday with the three AI stories that actually matter. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the AI Security quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Frequently asked questions
Is a strong system prompt enough to secure an assistant?
No. Authentication, authorization, input and output handling, tool limits, and incident response remain necessary parts of the application.
Keep learning
Related guides
More guides picked for this topic