Find tools › Open-source projects › Test & monitor AI apps

promptfoo
Open-source project · by promptfoo
A free tool that tests your AI prompts and chatbots automatically and flags bad or unsafe answers.
Some setupPopular4,832,983 installs a month
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Compare answers from several AI models side by side
- Check a company chatbot for unsafe or leaky replies
- Catch worse answers after you change a prompt
How to start
Install Node.js, then in a terminal run
npm install -g promptfoo.Create a starter project with promptfoo init --example getting-started.
Add your AI provider key, then run promptfoo eval and promptfoo view to see results.
Before you install
- Checked by a personWe read what it does before listing it.
- License: MITYou are allowed to use it under these terms.
- Last updated 2026-11-18
More detail
It is a command-line tool and library for testing prompts, agents, and RAG applications, featuring automated evaluations, red teaming, and model comparison.
It helps developers move away from trial-and-error by providing metrics, security vulnerability scans, and CI/CD checks for LLM applications. Reach for it when you need to rigorously test output quality or pentest an AI app before shipping.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map