Find tools › Open-source projects › Keep data private

sglang
Open-source project · by sgl-project
Server software that runs AI models fast for many users at once on your own graphics cards.
For developersPopular911,304 installs a month
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Host an open AI model for your whole team
- Serve AI answers faster and cheaper on your own hardware
- Power a company chatbot without a cloud AI provider
How to start
You need a machine with a strong NVIDIA or AMD graphics card.
In a terminal, run
pip install sglang.Follow the docs to launch a model server and connect your apps to it.
Before you install
- Checked by a personWe read what it does before listing it.
- License: Apache-2.0You are allowed to use it under these terms.
- Last updated 2026-11-18
More detail
SGLang is a high-performance serving framework for large language models and multimodal models designed to deliver low-latency and high-throughput inference.
It solves the bottleneck of slow and expensive model inference by providing advanced features like RadixAttention prefix caching and prefill-decode disaggregation, making it ideal when you need to serve open-weight models at scale.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map