If you're shipping anything with an LLM in the request path (a support bot, an agent that calls tools, a RAG pipeline reading untrusted documents) you eventually have to answer "did we actually test this for prompt injection, or did we just eyeball it." There are a handful of open source tools that can help, and they are not interchangeable. Picking the wrong one wastes a day; picking none wastes a lot more than that.
I went through the five tools that come up most often, ran them where I could, and read the source/docs closely where I couldn't. Here's what each one actually does, where they overlap, and where they don't.
Update, Aug 2026: Microsoft archived PyRIT on GitHub on March 27, 2026. It's read-only now, no commits, no releases, no issue triage. I've kept the section below for context since a lot of existing guides and tutorials still point people to it, but if you're picking a tool today, skip to the "which one, for what" section, or read the dedicated post on what to use instead.
The two layers people conflate
Most confusion here comes from mixing up two different things you can test:
