Self-hosted LLM runtime
Short answer: Open-source LLM runtime tools you can self-host.
Last updated: · No tax figures on this site, so no GOV.UK check applies; tool details come from each project's public GitHub repository · Markdown version
Ollama: Run LLMs locally with a single command.
LocalAI: OpenAI-compatible API server for local models.
vLLM: High-throughput LLM serving engine.
llama.cpp: LLM inference in C/C++, runs on CPU.