Find tools › Open-source projects › Test & monitor AI apps

issue-rag
Open-source project · by zaxiu88888
A tool that tests how accurately AI answers technical questions using real issue reports.
Some setup
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Harvest maintainer replies from GitHub issues
- Measure accuracy gains from retrieval pipelines
- Audit AI judge scoring against human reviews
How to start
Install dependencies with
pip install -e ".[models,dev]"Harvest issue titles and maintainer comments from GitHub
Build the search corpus and review generated test questions
Run evaluations using local model weights or external provider keys
Before you install
- Found automaticallyNot reviewed by a person yet. Check it before installing.
- License: MITYou are allowed to use it under these terms.
- Last updated 2026-11-04
More detail
A system to harvest GitHub issues, test document retrieval pipelines against maintainer answers, and audit LLM grader accuracy.
AI models often hallucinate outdated or incorrect library version details when answering support questions. Use this tool to build a realistic test set from actual maintainer replies and measure how much retrieval actually improves answer quality.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-11-07. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map