Find tools › Open-source projects › Keep data private

llama.cpp

Open-source project · by ggml-org

The engine behind many local AI apps; it runs AI models on ordinary computers without the cloud.

Some setupVery popular

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Chat with a small AI model on a laptop without internet
  • Run a local AI server that other apps can use
  • Test how fast different models run on your hardware

How to start

  1. Download a ready-made build from the Releases page, or follow the project website's instructions.

  2. In a terminal, run llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF to download and chat with a small model.

  3. Use its serve command instead to run a local server with a built-in web chat.

Good to know. Lower-level than Ollama: you pick model files and settings yourself, and getting the best GPU speed may require the right build or compiling from source.

Before you install

  • Checked by a personWe read what it does before listing it.
  • License: MITYou are allowed to use it under these terms.
  • Last updated 2026-11-18

More detail

A C/C++ program that runs AI language and vision models on ordinary hardware (CPU, Apple Silicon, NVIDIA/AMD/Intel GPUs) and is the engine many other local-AI tools build on.

Reach for it when you want direct, fine-grained control over running a model locally, or a lightweight OpenAI-compatible server with a built-in web UI (llama serve). Its compressed (quantized) model formats let larger models fit in less memory.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map