Find tools › Open-source projects › Keep data private

Xolotl

Open-source project · by Indras-Mirror

Run massive AI models on a single gaming computer by splitting their memory across fast storage drives and memory.

For developers

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Run massive MoE models locally on a single desktop
  • Serve OpenAI- and Anthropic-compatible endpoints from home
  • Power local coding agents without cloud subscription costs

How to start

  1. Clone the repository and run bash tools/xolotl/fetch-llama.sh to grab core dependencies.

  2. Compile the project binaries with CMake and ninja -C build-xolotl targeting your card architecture.

  3. Download the quantized model files and run python3 tools/v41/serve/serve_ds41.py with your chosen configuration.

Good to know. Experimental research project requiring high-end hardware (NVIDIA GPU with 24 GB VRAM, 90+ GB RAM, fast NVMe), and currently requires Linux and compiling from source.

Before you install

  • Found automaticallyNot reviewed by a person yet. Check it before installing.
  • License: MITYou are allowed to use it under these terms.
  • Last updated 2026-11-18

More detail

An inference engine and local server designed to run massive mixture-of-experts models across graphics memory, system RAM, and fast solid-state drives.

Massive AI models usually require expensive data center clusters. Reach for this to run 500B+ parameter models locally on a single gaming computer with 24 GB of graphics memory.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-10-18. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map