Protect AI's ModelScan and NB Defense: Open-Source AI Supply-Chain Scanning

Most AI security attention goes to the model’s behavior — jailbreaks, prompt injection, toxic output. But there’s an earlier, quieter risk: the artifacts you load before the model ever generates a token. A serialized model file or a shared notebook can carry executable code, leaked secrets, or PII straight into your environment. Protect AI publishes two open-source tools aimed squarely at that supply-chain layer: ModelScan and NB Defense. This review covers what each one actually does, drawn from their own repositories and documentation, and where the honest limits are.

A note on corporate context up front, because it matters for accuracy: Protect AI agreed to be acquired by Palo Alto Networks in 2025. Both tools discussed here remain published as open-source projects under the Apache 2.0 license at the time of writing, and this review treats them as the standalone open-source tools they are rather than as part of any commercial platform. Verify current maintenance status against the repositories before adopting either.

ModelScan: scanning for model serialization attacks

ModelScan is a static-analysis scanner for machine-learning model files. Its target is a specific, real, and under-appreciated threat class: model serialization attacks. When a model is saved, the serialization format can embed code that executes automatically when the file is later loaded. Per Protect AI, the consequences of a malicious model include credential theft, data theft, data poisoning, and model poisoning — all triggered by the simple act of loading a file you downloaded.

The reason this matters is that loading a model is routine and trusted. Teams pull weights from public hubs, internal registries, and colleagues’ machines dozens of times a day. The serialization formats that underlie much of the Python ML ecosystem are, by design, capable of executing arbitrary code on deserialization. ModelScan exists to inspect those files before you load them.

What it scans

Per the ModelScan README, the tool reads files byte-by-byte and identifies unsafe operators and code patterns without executing the model. Supported formats include:

• Serialized Python formats: PyTorch (torch.save/torch.load), and classic ML libraries such as scikit-learn and XGBoost via the common Python serialization libraries (including cloudpickle, dill, and joblib)
• TensorFlow SavedModel (Protocol Buffer) format
• Keras H5 and Keras V3 formats

Installation is a single pip install modelscan; TensorFlow and H5 support pull in optional dependencies (pip install 'modelscan[tensorflow,h5py]'). It requires a supported Python version (3.9–3.12 per the documentation at the time of writing).

What a scan looks like

ModelScan is a command-line tool. You point it at a file or a directory:

modelscan -p /path/to/model_file

Findings are ranked by severity into CRITICAL, HIGH, MEDIUM, and LOW tiers. Reporting can be emitted to the console or as JSON (-r json -o results.json), which is the integration point that matters: JSON output is what lets you wire ModelScan into a CI gate or a registry-admission check rather than reading console output by hand. The tool also exposes meaningful exit codes — 0 for a clean scan, 1 when issues are found, and distinct codes for scan errors and usage problems — so a build step can branch on the result.

Honest limits

ModelScan is a static, signature-and-operator scanner, not a sandbox and not a behavioral analyzer. Two consequences follow.

First, it detects known unsafe operators and patterns. A genuinely novel obfuscation that doesn’t match its detection logic can slip through; the README itself frames format support as something that “will be expanding continually,” which is the honest posture for a tool in this category. Treat a clean ModelScan result as “no known-unsafe operators found,” not “provably safe to load.”

Second, it covers the formats it covers. If your artifact is in a format ModelScan doesn’t parse, it can’t assess it — it will report that it skipped unsupported files (use --show-skipped to see them) rather than giving false assurance. That transparency is good behavior, but it means you need to confirm your formats are actually in scope.

The right mental model: ModelScan is a cheap, fast tripwire for the most common and most dangerous class of model-file attack. It belongs in the path between “downloaded a model” and “loaded it in production,” not as your only defense.

NB Defense: security scanning for Jupyter notebooks

NB Defense addresses a different artifact: the Jupyter notebook. Notebooks are where a lot of ML work actually happens, and they are notorious for accumulating risk — hard-coded API keys in a cell, real customer data pasted in for a quick test, a dependency with a known CVE, a license-incompatible package. Because notebooks mix code, output, and prose, that risk is easy to commit and hard to spot in review.

Per Protect AI, NB Defense scans notebooks and their dependency files for four concerns:

• Secrets — API keys and credentials embedded in cells
• PII — personally identifiable information in code or output
• CVEs — known vulnerabilities in declared dependencies
• License issues — license-compliance problems in dependencies

How you run it

NB Defense ships in two forms: a command-line tool for scanning a notebook or a project directory, and an SDK that backs a JupyterLab extension, so findings can surface inside the notebook environment as you work rather than only in a separate scan step. It installs via pip and is published under the Apache 2.0 license.

The CLI form is the one to lean on for automation — running NB Defense over a repository of notebooks as part of CI catches the “oops, a key got committed” class of mistake before it reaches a shared branch. The JupyterLab extension is the shift-left complement: it puts the same checks in front of the person writing the notebook, which is where the secret or the pasted PII originates.

Honest limits

Secret and PII detection is pattern-and-heuristic work, and like all such tooling it lives on a precision/recall trade-off. Expect both false positives (a string that looks like a key but isn’t) and the possibility of missed secrets in unusual formats. The CVE check is only as current as the vulnerability data behind it and only sees dependencies you’ve actually declared. None of this is a knock specific to NB Defense — it’s the nature of the category — but it means NB Defense is a strong first-pass gate, not a guarantee that a notebook is clean.

Where these two tools fit

ModelScan and NB Defense sit at a different layer from the runtime guardrails and red-team scanners we cover elsewhere. They are AI supply-chain tools: they inspect the artifacts (model files, notebooks, dependencies) that flow into your pipeline, before any inference happens.

• For runtime input filtering and prompt-injection detection, see our Lakera Guard review and Guardrails AI review.
• For adversarial probing of a deployed model’s behavior, see our deep dives on garak and PyRIT.
• For how to weigh any of these against each other, see our AI security tool evaluation framework.

The supply-chain layer is the one teams most often skip, precisely because loading a model or running a notebook feels safe. It isn’t inherently safe, and these two tools close a real gap at essentially zero cost.

Verdict

ModelScan is an easy yes. It is a small, focused, free tool that addresses one of the most concrete and dangerous AI supply-chain risks — code execution on model load — and it produces machine-readable output suited to CI gating. Adopt it as a pre-load check for any model artifact you didn’t build yourself, understanding that it catches known-unsafe patterns rather than proving safety.

NB Defense is a strong fit for any team whose real work lives in notebooks. Its four-pillar scan (secrets, PII, CVEs, licenses) maps directly to the mistakes notebooks invite, and the JupyterLab extension puts detection where the mistakes are made. Treat its findings as a high-value first pass, not a clean bill of health.

Both are open source, both are narrow by design, and both are worth wiring into your pipeline. Their narrowness is the point: they do one layer well and don’t pretend to be a whole AI security program. Pair them with runtime and red-team tooling for full coverage.

For broader AI security tool comparisons across the stack, bestaisecuritytools.com ↗ maintains updated benchmark data.