Find tools › Open-source projects › Spend less on AI

LLMLingua

Open-source project · by microsoft

A code library that shortens the text you send to AI, cutting cost while keeping the meaning.

For developersWell known36,730 installs a month

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Shrink long documents before sending them to a paid AI model
  • Fit more background text into one AI request
  • Lower the monthly bill of an AI feature you run

How to start

  1. Have Python installed on your computer.

  2. In a terminal, run pip install llmlingua.

  3. Add it to your Python code using the examples in the README to compress prompts.

Before you install

  • Checked by a personWe read what it does before listing it.
  • License: MITYou are allowed to use it under these terms.
  • Last updated 2026-11-09

More detail

It compresses prompts and KV-caches using smaller companion models to reduce token counts and lower API costs.

It solves the problem of high API costs and context window limits when sending massive prompts to large language models. Reach for it when you want to squeeze longer documents or RAG contexts into fewer tokens without retraining your main model.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map