Promptfoo
CLI evaluation tool and test runner for LLM application security, automated red teaming, and CI/CD security pipeline gating.
Technical Architecture & Overview
Promptfoo is an open-source testing tool for LLMs, RAG applications, and agentic workflows. It includes over 150 automated red-team plugins aligned with the OWASP GenAI LLM Top 10, covering prompt injection, PII disclosure, SSRF through tool calls, and jailbreaks.
Targeted Technical Use Cases
Continuous security testing and prompt regression checks in GitHub Actions and GitLab pipelines.
Evaluation & Trade-offs
Core Strengths
- +Runs locally using declarative YAML configuration files.
- +Dedicated plugins for prompt injection, PII disclosure, SSRF through tool calls, and jailbreaks.
- +Supports custom assertions via Python, JavaScript, or LLM-graded rubrics.
Trade-Offs & Limitations
- -Some red-team generation and grading plugins use Promptfoo's hosted service by default; fully air-gapped operation requires disabling remote generation.
- -Complex multi-agent simulations require custom test scripting.
Defensive Security Application
Catching prompt regressions, unauthorized tool usage, and sensitive data leakage during pull request reviews.
Frequently Asked Questions
What is Promptfoo?→
Promptfoo is an open-source testing tool for LLMs, RAG applications, and agentic workflows. It includes over 150 automated red-team plugins aligned with the OWASP GenAI LLM Top 10, covering prompt injection, PII disclosure, SSRF through tool calls, and jailbreaks.
What is Promptfoo used for?→
Continuous security testing and prompt regression checks in GitHub Actions and GitLab pipelines.
What are the strengths of Promptfoo?→
- +Runs locally using declarative YAML configuration files.
- +Dedicated plugins for prompt injection, PII disclosure, SSRF through tool calls, and jailbreaks.
- +Supports custom assertions via Python, JavaScript, or LLM-graded rubrics.
What are the limitations of Promptfoo?→
- +Some red-team generation and grading plugins use Promptfoo's hosted service by default; fully air-gapped operation requires disabling remote generation.
- +Complex multi-agent simulations require custom test scripting.
How is Promptfoo used defensively?→
Catching prompt regressions, unauthorized tool usage, and sensitive data leakage during pull request reviews.