Find tools › Open-source projects › Spend less on AI

LLMLingua
Open-source project · by microsoft
A code library that shortens the text you send to AI, cutting cost while keeping the meaning.
For developersWell known36,730 installs a month
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Shrink long documents before sending them to a paid AI model
- Fit more background text into one AI request
- Lower the monthly bill of an AI feature you run
How to start
Have Python installed on your computer.
In a terminal, run
pip install llmlingua.Add it to your Python code using the examples in the README to compress prompts.
Before you install
- Checked by a personWe read what it does before listing it.
- License: MITYou are allowed to use it under these terms.
- Last updated 2026-11-09
More detail
It compresses prompts and KV-caches using smaller companion models to reduce token counts and lower API costs.
It solves the problem of high API costs and context window limits when sending massive prompts to large language models. Reach for it when you want to squeeze longer documents or RAG contexts into fewer tokens without retraining your main model.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map