Find tools › Open-source projects › Keep data private

vllm

Open-source project · by vllm-project

Software that runs open AI models very fast on powerful graphics cards for many users at once.

Some setupVery popular2,657,706 installs a month

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Serve an open model to a whole team from one server
  • Replace a paid AI service with your own hardware
  • Handle many AI requests at once without slowing down

How to start

  1. Use a machine with a strong graphics card and Python installed.

  2. In a terminal, run uv pip install vllm.

  3. Start its server with a model name and connect apps to its OpenAI-style address.

  4. For one person on a laptop, Ollama or llama.cpp are simpler.

Good to know. Built for server-grade GPUs and Python environments; it is not a point-and-click app. For a single user on a laptop, Ollama or llama.cpp are simpler.

Before you install

  • Checked by a personWe read what it does before listing it.
  • License: Apache-2.0You are allowed to use it under these terms.
  • Last updated 2026-11-18

More detail

A Python library and server from UC Berkeley's Sky Computing Lab for running open-weight models at high speed on GPUs, serving many requests at once through an OpenAI-compatible API.

Reach for it when you need to serve a model to many users or apps from your own GPU servers and care about throughput and memory efficiency. It is aimed at production serving rather than casual desktop chatting.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map