llama.cpp

Local models served by llama.cpp's own server (llama-server). Nothing leaves your computer and there is no cost per call.

Set it up#

1. Start llama-server with the model you want.
2. In Settings → Model Hub → Providers, select the llama.cpp tile.
3. Check Server address. It starts as http://127.0.0.1:8080/v1, the server's default; change it if you gave the server another port or host.
4. Select Test.

There is no key, so nothing is kept in the keychain.

Models and prices#

The Hub offers the models the server lists. Calls are free: the ledger records the tokens with a cost of $0, and per-call and monthly limits never stop a call on this provider.

Reasoning#

There is no reasoning field of its own. The Reasoning option on a model's row is sent as reasoning_effort, as for any OpenAI-compatible endpoint.

Related#

• Add a provider
• Privacy and costs