One address for your local AI.

Several good models, one app at a time

What a routing decision looks like

Every request produces a record like these. The scenarios below are scripted illustrations using the three models of a recorded test run — not live inference.

Read the full pipeline — requirements, hard filters, ranking, continuity — in How it works.

Three steps to a first routed request

1. Install ModelSaar

   Python 3.10 or newer on macOS, Linux or WSL2. The one-line installer uses uv, pipx or pip --user — whichever you have — and pins one candidate commit.

   curl -fsSL https://modelsaar.com/install.sh | sh

2. Start the gateway

   The installer has already run modelsaar init: it found Ollama on 127.0.0.1:11434 (an inference server with at least one model must already be running), wrote a private config and keys, and listed the models it discovered. Now start the gateway and leave it running.

   modelsaar serve

   Installed from source instead? Run modelsaar init first (once; it refuses to overwrite an existing config), then modelsaar serve.

3. Connect an app

   Point the app at the gateway and pick auto as the model. Print the API key in a terminal, then paste the printed line (it starts with msk_) into the app's key field:

   head -1 ~/.modelsaar/keys/api_keys

   Base URL:  http://127.0.0.1:8484/v1
   Model:     auto
   API key:   msk_…  (the line printed by the command above)

   One command per app prints its exact settings: modelsaar client-config open-webui, modelsaar client-config opencode, modelsaar client-config openclaw or modelsaar client-config hermes.

Verify from a second terminal with modelsaar test: it first identifies the gateway on the configured port (an unrelated service there is reported as a collision and receives no key), lists the models the gateway advertises and sends one chat completion through auto, printing which deployment answered and the request id — the routing attribution that shows ModelSaar handled it. Full details, including Docker and remote hosts, in Get started.

What works today, and how we know

Compatibility is a tested property of an exact client, protocol and backend combination. Each row says what kind of evidence backs it.

live ran against a real backend; fixture tests are the repository's test suite against a controlled fake backend (run locally for every number on this site; the same suite runs in CI on push, and a CI run is reported only once one exists for the revision); documented means the configuration was written from the client's own documentation and has not been exercised. Product names identify compatible software; no partnership or endorsement is implied. Details per client and backend: Integrations.

Measured, not promised

In the recorded run of 2026-10-11 (three models on one machine, 32 held-out tasks, 400-token budget), automatic routing and the strongest single model each scored 30 of 32; automatic routing completed 32 requests versus 31 (the image task the fixed model cannot take), recorded zero capability violations versus one, and had a p95 end-to-end latency of 7.958 s versus 4.760 s because two requests went to a slower thinking model. Routing overhead — the decision itself, separate from the response — was 0.98 ms at p95.

That is one run on one machine with a small task set; the confidence intervals overlap. Read the methodology, numbers and limitations, or regenerate every aggregate from the raw per-request records.

Try it on the models you already have

Python and a running backend are all you need for a first routed request. If something does not work, the issue templates ask for exactly the decision record and doctor output that make it fixable.