Find tools › Open-source projects › Spend less on AI

tokenspeed
Open-source project · by lightseekorg
Run large language models at ultra-fast speeds on high-end GPUs for AI agent workloads.
Some setupWell known141 installs a month
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Serve open-source LLMs with high throughput
- Power latency-sensitive multi-step AI agent loops
- Host models on enterprise NVIDIA and AMD accelerators
How to start
Check the installation requirements in the official getting started documentation.
Configure your GPU drivers and runtime environment for your accelerator platform.
Launch the AsyncLLM server using the recommended model recipe for your deployment.
Before you install
- Found automaticallyNot reviewed by a person yet. Check it before installing.
- License: MITYou are allowed to use it under these terms.
- Last updated 2026-11-18
More detail
A high-performance open-source LLM inference engine optimized for agentic workloads on modern accelerator hardware.
Standard inference runtimes can face latency bottlenecks during agent loops. Reach for it to maximize generation throughput and token generation speed on enterprise GPUs.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-10-15. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map