Reviews of AI security products and platforms.
Hands-on reviews of AI security products and platforms. We deploy them against real attack libraries, measure detection rates and false positives, and publish the numbers — including the ones the vendor would rather we didn't.
OWASP LLM Top 10 Mitigation Guide: Controls for Every Risk Category (2025 Edition)
A practitioner's OWASP LLM Top 10 mitigation guide covering all ten 2025 risk categories — prompt injection through unbounded consumption — with concrete controls, tooling pointers, and residual-risk notes.
Earlier entries
-
Patronus AI Review: Automated LLM Evaluation and Guardrails
A review of Patronus AI's evaluation platform — the Lynx hallucination model, the Glider custom evaluator, the built-in judge and safety evaluators, and how its self-serve API fits into an AI security stack.
-
Protect AI's ModelScan and NB Defense: Open-Source AI Supply-Chain Scanning
A hands-on review of Protect AI's two best-known open-source tools — ModelScan for model serialization attacks and NB Defense for Jupyter notebooks. What they actually detect, how to run them, and where their limits are.
-
Robust Intelligence (Now Cisco AI Defense): What the Platform Actually Covers
A conservative review of Robust Intelligence — the AI security pioneer now part of Cisco AI Defense. Algorithmic red teaming, AI Validation, model file scanning, and runtime AI Protection, with the public/gated line clearly marked.
-
PyRIT Deep Dive: Microsoft's AI Red Teaming Framework in Practice
A long-form review of PyRIT, Microsoft's open-source AI red teaming framework. Its orchestrator/target/converter/scorer/memory architecture, multi-turn attack support, result persistence, and where it fits versus garak — described from the project's own docs.
-
Garak Deep Dive: Architecture, Probes, and Operating the NVIDIA LLM Scanner
A hands-on, long-form review of garak — NVIDIA's open-source LLM vulnerability scanner. How its probe/detector/generator/buff architecture actually works, which model backends it speaks, what its reports contain, and how to operate it without drowning.
-
Giskard Review: Open-Source Testing and Evaluation for LLM and RAG Apps
A long-form review of Giskard, the open-source Python library for testing AI systems. Its automated Scan for LLM vulnerabilities, the RAGET RAG-evaluation toolkit, the giskard.Model wrapper, and where it fits beside red-team scanners like garak and PyRIT.
-
How to Evaluate AI Security Tools Without Getting Fooled
AI security tool demos are optimized for best-case scenarios. A rigorous evaluation requires adversarial test cases, production-realistic inputs, and honest accounting of false positive costs. Here's the framework.
-
PyRIT: Microsoft's AI Red Teaming Tool in Security Workflows
PyRIT is Microsoft's open-source AI red teaming framework. Built for enterprise security teams, it has better CI/CD integration than research-first tools. The tradeoff is probe breadth.
-
Guardrails AI: Output Validation That Doesn't Require Retraining
Guardrails AI provides a validation layer for LLM outputs — checking format, structure, and content without touching the model. The validator library is extensive. The performance overhead is manageable with the right configuration.
Trusted by researchers across the AI security community
AI Sec Reviews is part of a 26-site editorial network covering adversarial ML, AI governance, defensive tooling, and ops engineering — all open access.
AI Sec Reviews — in your inbox
Reviews of AI security products and platforms. — delivered when there's something worth your inbox.
No spam. Unsubscribe anytime.