Find tools › Open-source projects › Keep data private

llamAmpere
Open-source project · by JakeATX
Run large language models much faster on your home RTX 3090 or 3090 Ti graphics card.
Some setup
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Run large language models locally on consumer Ampere GPUs
- Speed up speculative decoding using custom GPU kernels
- Process extremely long text contexts up to 262k tokens
How to start
Clone the repository using
git clone -b v0.4 https://github.com/JakeATX/llamAmpere.gitBuild the server binary using CMake with CUDA enabled
Download and load a compatible Qwen model from Hugging Face
Launch the server to run your local model
Before you install
- Found automaticallyNot reviewed by a person yet. Check it before installing.
- License: MITYou are allowed to use it under these terms.
- Last updated 2026-11-18
More detail
llamAmpere is a high-performance fork of llama.cpp optimized specifically to speed up local model inference on consumer Ampere graphics cards.
Standard local LLM runners can run slowly with large contexts. This fork uses optimized GPU kernels and custom KV cache options to achieve much higher generation speeds on RTX 3090 and 3090 Ti GPUs.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-30. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map