Find tools › Open-source projects › Keep data private

llama.cpp
Open-source project · by ggml-org
The engine behind many local AI apps; it runs AI models on ordinary computers without the cloud.
Some setupVery popular
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Chat with a small AI model on a laptop without internet
- Run a local AI server that other apps can use
- Test how fast different models run on your hardware
How to start
Download a ready-made build from the Releases page, or follow the project website's instructions.
In a terminal, run
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUFto download and chat with a small model.Use its serve command instead to run a local server with a built-in web chat.
Before you install
- Checked by a personWe read what it does before listing it.
- License: MITYou are allowed to use it under these terms.
- Last updated 2026-11-18
More detail
A C/C++ program that runs AI language and vision models on ordinary hardware (CPU, Apple Silicon, NVIDIA/AMD/Intel GPUs) and is the engine many other local-AI tools build on.
Reach for it when you want direct, fine-grained control over running a model locally, or a lightweight OpenAI-compatible server with a built-in web UI (llama serve). Its compressed (quantized) model formats let larger models fit in less memory.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map