MIT-licensed CLI for LLM evals and automated red teaming, with 71 documented vulnerability plugins, 34 attack strategies and a quote-only enterprise tier.
Engineers who want adversarial testing in the same place as their prompt evaluations and running in CI, since the docs enumerate 71 vulnerability plugins and 34 attack strategies and ship pipeline guides for GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines, Bitbucket Pipelines, Travis CI and AWS CodeCommit. The community edition runs from the command line against a local SQLite store, which suits teams that need test artefacts to stay on infrastructure they control. It also covers model file inspection through ModelAudit, which documents scanners for pickle, SafeTensors, GGUF/GGML, ONNX, Keras, TensorFlow, PyTorch, JAX/Flax, XGBoost and PMML among others.
Self-hosting the community edition is explicitly not production-grade: the docs say it 'is not recommended for production use cases', ship no built-in authentication or SSO, provide no RBAC or multi-team support, and cannot scale horizontally because jobs live in each server's memory, so more than one replica produces 'Job not found' errors. The free tier is capped at 10,000 red team probes per month, and 11 of the 34 strategies, including Best-of-N, GCG, Composite Jailbreaks, GOAT and Hydra Multi-turn, are marked as using remote inference on that edition, so fully isolated red teaming means the on-premise tier with its dedicated runner. Enterprise and on-premise are both quote-only with no published numbers, and no detection-rate or latency figures are published for the guardrails checks or ModelAudit scanners.
Pros
Cons
Yes, under the MIT licence. The repository README states that promptfoo remains open source and MIT licensed, and GitHub reports MIT as the detected licence. The free Community edition includes all evaluation features, every documented provider and integration, vulnerability scanning and CI/CD use, plus red teaming capped at 10,000 probes per month. Team controls, RBAC, API access and an SLA sit on the paid tiers.
Most but not all of it. The strategy table lists 34 strategies and marks 11 as using remote inference in the Community edition, including Best-of-N, GCG, Composite Jailbreaks, Likert-based Jailbreaks, Meta-Agent Jailbreaks, GOAT, Goblin Multi-turn, Hydra Multi-turn, Citation, Authoritative Markup Injection and Audio Encoding. Enterprise On-Prem is the documented route to a dedicated runner that keeps scans inside your own network perimeter.
It gives a Docker image at ghcr.io/promptfoo/promptfoo, a web UI and API on port 3000, and eval storage in a local SQLite file that you must mount as a volume or lose when the container is removed. The docs say outright it is not recommended for production: no built-in authentication or SSO, no RBAC or multi-team support, and no horizontal scaling because in-memory jobs break across replicas.
No. The red team docs enumerate 71 vulnerability plugins and 34 strategies, and the ModelAudit docs list scanners for formats including pickle, SafeTensors, GGUF, ONNX, Keras, PyTorch, JAX and XGBoost, but no accuracy, recall or false-positive figures are published for either. ModelAudit's own page notes that unknown formats receive only generic binary scanning and that encrypted 7-Zip archives may conceal content from inspection.
Facts sourced from the vendor’s own site, last verified 2026-10-10.
The Weekly Source
One email a week. New listings, repriced tools, and the ones we would skip - including this one if its pricing changes.
