Find tools › Open-source projects › Keep data private

inferssd

Open-source project · by wici-ai

Runs huge artificial intelligence models from your computer's solid-state drive when your memory is too small.

Some setupWell known

What it costs

Free and open source; you may pay for the AI model it uses.

Use it to

  • Run massive MoE models on standard desktop hardware
  • Serve an OpenAI-compatible web API from your local drive
  • Chat with large open models directly in your terminal

How to start

  1. Run the installation script in your terminal: curl -fsSL https://github.com/wici-ai/inferssd/releases/latest/download/install.sh | sh

  2. Start terminal chat or serve a model with inferssd serve -hf mrfakename/DeepSeek-V4.1-Flash-1017-GGUF

  3. Open http://127.0.0.1:8080 in your web browser to start chatting

Good to know. Requires hundreds of gigabytes of fast SSD storage and puts heavy read traffic on your drive, and the server API mode lacks built-in authentication.

Before you install

  • Found automaticallyNot reviewed by a person yet. Check it before installing.
  • License: Apache-2.0You are allowed to use it under these terms.
  • Last updated 2026-11-18

More detail

InferSSD is a local model engine based on llama.cpp that streams large Mixture-of-Experts AI models directly from your computer's SSD.

It enables running AI models larger than your available RAM or graphics memory without buying high-end server hardware. Reach for it when you want to run massive open models locally on standard consumer computers.

Similar tools

Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-11-12. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map