Alternatives to vLLM

Ordered by how much they overlap with vLLM, then by score.

Alternatives, ranked

  1. Ollama

    Runtime to download and run large language models locally, with a CLI and a REST API.

    • Higher score
    • Runs on more platforms
    • No shared features
    Updated 62 days ago
    5 93.5 = Compare
  2. llama.cpp

    C/C++ inference engine for LLMs on CPUs and GPUs, using the GGUF format and quantization.

    • Released more recently
    • Higher score
    • Runs on more platforms
    • No shared features
    Updated 61 days ago
    6 93.0 = Compare
  3. LocalAI

    Self-hosted, OpenAI-compatible API that runs text, image and audio models on local hardware.

    • Higher score
    • Runs on more platforms
    • No shared features
    Updated 62 days ago
    64 85.2 = Compare
  4. LM Studio

    Discover, download and run local LLMs offline on Mac, Windows and Linux with an intuitive desktop interface and local OpenAI-like server.

    • Runs on more platforms
    • No shared features
    Stale: 646 days without a release
    367 40.0 = Compare