Promptfoo is an open-source platform for testing, evaluating, red teaming, and securing LLM applications and AI workflows.

Who should use Promptfoo?

It is best for developers and AI teams that need repeatable model tests, security checks, RAG evaluation, and CI/CD quality gates.

Can Promptfoo test prompt changes?

Yes. Promptfoo can compare prompts and model configurations against test cases to identify regressions and improvements.

Does Promptfoo support red teaming?

Yes. It includes red-teaming and vulnerability-testing workflows designed to probe AI applications for common security failures.

The Community version is free and includes core evaluation capabilities, while enterprise and on-premise options add commercial features.

What should teams evaluate?

Review test coverage, model costs, security requirements, CI integration, data handling, and enterprise support needs before deployment.

Promptfoo

Name: Promptfoo
Brand: Promptfoo

Open-source LLM testing, red teaming, guardrails, and evaluation framework.

Visit Website

What is Promptfoo?

TL;DR

Promptfoo is an open-source LLM testing, red teaming, guardrails, and evaluation platform that helps developers compare models, test RAG pipelines, scan vulnerabilities, and add AI quality checks to CI/CD workflows.

Evaluate prompts, models, RAG pipelines, and LLM outputs
Run red teaming and vulnerability scans for AI apps
Use locally, self-hosted, managed cloud, or on-premise

Explore with AI

About

Promptfoo is an open-source LLM testing, red teaming, guardrails, and evaluation platform for developers building AI applications. It helps teams test prompts, compare models, evaluate RAG pipelines, scan for vulnerabilities, and add structured quality checks to AI development workflows. The product is useful for SaaS founders and technical teams because AI apps can fail through hallucinations, prompt injection, unsafe outputs, jailbreaks, poor retrieval, and regression after small prompt changes. Promptfoo gives builders a way to test these behaviors locally, in CI/CD, or through enterprise workflows. Promptfoo's pricing page describes a free Community version with all LLM evaluation features, all model providers and integrations, red teaming limits, vulnerability scanning, and self-hosting. Enterprise and on-premise options add collaboration, monitoring, dashboards, SSO, API access, managed cloud deployment, and support. For FutureStack, Promptfoo belongs in Developer Tools because LLM security and testing are critical for production AI. It is best for developers, founders, and AI teams that need to test, secure, and monitor AI applications before users break them.

Use Cases

LLM Regression Testing

Run repeatable prompt and model tests in development or CI/CD to catch quality regressions before release.

AI Red Teaming

Probe applications for jailbreaks, prompt injection, unsafe outputs, and other common LLM security vulnerabilities.

RAG Evaluation

Test retrieval and generation behavior across datasets to identify relevance, grounding, and response-quality failures.

AI Security Gates

Add vulnerability scans and structured checks to the software delivery process before AI features reach users.

Key Features

Evaluate prompts, models, RAG pipelines, and LLM outputs
Run red teaming and vulnerability scans for AI apps
Use locally, self-hosted, managed cloud, or on-premise
Integrates into developer workflows, CI/CD, and security processes

Featured Tools

Wispr Flow

Voice-First Dictation

We may earn commissions from links to support our work. Learn more.