Technical GUIDE
LLM Vulnerability Scanners: garak and PyRIT
NVIDIA garak is an open-source scanner for security testing systems that accept prompts and return text, using selected probes and detectors to examine defined failure modes.
On this page3 min read
Overview
PyRIT is a broader red-teaming framework that can run selected security scenarios and attacks. Neither tool’s results certify a model or application as secure; findings depend on the target, test selection, and configuration.
Deep Dive
NVIDIA garak is an open-source LLM vulnerability scanner whose documentation frames its purpose as testing the security of systems that take prompts and return text. It uses a configured generator to communicate with the target, probes to attempt a defined failure, attempts to record interactions, detectors to look for a particular failure signal, and evaluators to summarize results for probe-detector pairs. Security-oriented examples include prompt injection, data leakage, and other prompt-driven weaknesses in the configured target. A scan is only as relevant as its selected target, probes, and detector behavior.
Microsoft PyRIT is a more general red-teaming framework with scanner, GUI, and framework modes. Its current documentation describes targets, scenarios, attack techniques, memory, and flexible scorers. Teams can select security-related scenarios, such as web injection or system-prompt extraction, when those objectives fit the system under review. PyRIT also supports broader safety assessment, but that broader scope should not be confused with a security guarantee or a claim that every scenario is a vulnerability test. The operator needs to choose the objective and understand how the configured scorer judges outcomes.
Use these tools to produce evidence for a defined security review, not a universal pass/fail certificate. Test an authorized staging or assessment target with the integrations relevant to the real application. Inspect attempts behind a flagged result, reproduce meaningful findings, determine whether the issue affects the deployed workflow, and record the model or endpoint version and chosen test configuration. A scan with no findings means only that those configured tests did not trigger a detector under that run. It does not show that untested attack paths, tools, data sources, or application controls are secure.
Strategic Impact
Cost and budget
Architecture decisions drive performance and operating cost for years.
Clearer decisions
Technical education helps teams choose the right stack, not just the newest one.
Quality control
Better engineering choices reduce reliability incidents in production.
The Future of LLM Vulnerability Scanners: garak and PyRIT
Security testing tools will keep adding probes, targets, scenarios, and scoring options, but a larger catalog does not make coverage complete. Teams will still need a threat model that reflects their application, integrations, and data access. Automated results can help prioritize investigation and track regressions when test setup is recorded. Human review and application-level verification remain necessary before teams describe a finding or a clean run as evidence about security. Teams should revisit tests when endpoints, tools, or data pathways change, and keep security claims tied to the conditions actually assessed.
Real-World Implementation
A team runs garak’s selected prompt-injection probe against a staging endpoint and inspects the recorded attempts and detector result.
A security tester selects a garak data-leakage probe to check whether a configured target exposes information placed in its test context.
A red team chooses a PyRIT scenario and attack technique for a defined prompt-injection or leakage objective, then reviews the stored conversation and scoring outcome.
An engineer reproduces a confirmed weakness in the application path and adds the exact input and expected control behavior to a regression check.
Risks & Guardrails
Optimizing one benchmark can hide broader system weaknesses.
Infrastructure and maintenance costs are often underestimated.
Security and observability gaps can grow as systems become more complex.
Implementation Roadmap
Define latency, quality, and cost targets before implementation.
Benchmark under realistic load and data conditions.
Instrument monitoring for errors, drift, and user impact.
Prepare rollback and incident response paths before scaling.
Keep Exploring
Free newsletter
Keep up with AI in 3 minutes a day
One short email each weekday with the three AI stories that actually matter. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the LLM Vulnerability Scanners: garak and PyRIT quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Frequently asked questions
What is LLM Vulnerability Scanners: garak and PyRIT?
NVIDIA garak is an open-source scanner for security testing systems that accept prompts and return text, using selected probes and detectors to examine defined failure modes. PyRIT is a broader red-teaming framework that can run selected security scenarios and attacks. Neither tool’s results certify a model or application as secure; findings depend on the target, test selection, and configuration.
How does garak describe its primary testing purpose?
The garak documentation says its goal is testing the security of prompt-in/text-out systems.
What does a garak probe do in a security scan?
The docs describe probes as trying to exploit a weakness and elicit a failure.
What does a garak detector report?
A detector checks the recorded response for a defined phenomenon.
Which record helps a reviewer understand why a garak probe was flagged?
Garak attempts record interactions and reports include detailed attempt data.
Which PyRIT component can package datasets with attack techniques for a run?
Current PyRIT docs describe scenarios as packaging datasets with attack techniques.
Keep learning
Related guides
More guides picked for this topic