OpenCues ships an ollama provider that runs models fully on your machine: no API key, no per-token cost, and your text never leaves the device. The trade is latency, and the right answer is often a split: local for some surfaces, cloud for others.
The privacy case for local
With Ollama, inference never leaves your machine, which makes it the most private option and, in the project's own words, the one provider OpenCues will route prose-bearing surfaces through without a second thought. There's also a hard guarantee attached: a local request never falls back to a cloud provider, because that would silently break the privacy promise. If the local server is down, the call fails visibly instead.
The latency case for cloud
Cloud inference on the fast providers is sub-second warm; word cues need keystroke-rate latency. Local inference on a laptop GPU is a different regime: the validated default model (gemma4:e2b, about 1.4 GB of VRAM) runs a transform in roughly 2-3 seconds warm, with a one-time model-load tax of about 10 seconds on first use (pin the model in VRAM with OLLAMA_KEEP_ALIVE=-1 to avoid re-paying it). The docs are direct: not recommended for keystroke-latency word cues; best on the deliberate _ surface, where you asked and are willing to wait a moment.
What runs well locally, and one sharp edge
Setup is three steps: install Ollama, ollama pull gemma4:e2b, and point OPENCUES.md at it, either globally (llm-provider: ollama) or just for blanks (blanks-llm-provider: ollama), keeping cloud speed for cues. The sharp edge: OpenCues deliberately talks to Ollama's native API rather than its OpenAI-compatible endpoint, because the compatible endpoint gives no way to disable a thinking model's reasoning channel, and a thinking model left thinking returns empty output on OpenCues' small token budgets. So use llm-provider: ollama, not an endpoint override of the openai provider.
Related: LLM providers · Is OpenCues safe? · Cerebras vs Groq · Self-hosted AI writing assistant · GitHub repo