Find tools › Open-source projects › Keep data private

sglang

Open-source project · by sgl-project

Server software that runs AI models fast for many users at once on your own graphics cards.

For developersPopular911,304 installs a month

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Host an open AI model for your whole team
  • Serve AI answers faster and cheaper on your own hardware
  • Power a company chatbot without a cloud AI provider

How to start

  1. You need a machine with a strong NVIDIA or AMD graphics card.

  2. In a terminal, run pip install sglang.

  3. Follow the docs to launch a model server and connect your apps to it.

Good to know. Requires dedicated GPU hardware (such as NVIDIA or AMD accelerators) and advanced configuration for distributed clusters.

Before you install

  • Checked by a personWe read what it does before listing it.
  • License: Apache-2.0You are allowed to use it under these terms.
  • Last updated 2026-11-18

More detail

SGLang is a high-performance serving framework for large language models and multimodal models designed to deliver low-latency and high-throughput inference.

It solves the bottleneck of slow and expensive model inference by providing advanced features like RadixAttention prefix caching and prefill-decode disaggregation, making it ideal when you need to serve open-weight models at scale.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map