solstone's memory use: hosted vs local models

solstone runs AI to think about your journal, read your screen, and turn audio into text. by default the heavy reasoning runs off your machine — light on your own hardware — and you can opt into running models locally for maximum privacy. local models are memory-hungry, so it's worth knowing what to expect before you switch.

the default: the heavy AI runs off-machine

out of the box, solstone keeps the big memory consumer off your machine:

• thinking and screen-analysis — the reasoning over your journal, and describing what's on screen — run off-machine by default, through a cloud model (you pick how in the thinking app — either scout, where we cover it for you, or your own key). no large model sits in your RAM for these.
• transcription — turning audio into text — runs locally by default, but with a small on-device model: roughly 0.5 GB on apple silicon, ~3 GB on linux.

so a fresh install does not load a full local language or vision model. you only take on a large memory footprint if you deliberately switch the reasoning over to local.

opting into local models: maximum privacy, heavy

running the reasoning and vision model locally means nothing leaves your machine — but it's heavy. on apple silicon the on-device model needs about 13 GB of free memory (a single model serves both vision and thinking). on a 16 GB mac that leaves little room, and whatever else you have open is competing for the same memory.

solstone guards against the worst case before it loads anything:

• apple silicon — it checks your free memory first and won't activate the local model if it won't fit, pointing you to an off-machine option instead.
• linux — local models need a supported GPU here. without one, solstone won't activate a local model and points you to an off-machine option instead — and, as on apple silicon, it won't load a model that won't fit in memory.

(these are relatively new safeguards — make sure you're on a current version, below.)

choosing in the thinking app

open solstone's web ui (by default http://localhost:5015) and go to the thinking app. it lays out how sol can think as a few lanes, one active at a time:

• scout — free thinking while you're an early tester; we cover it for you.
• BYO cloud — your own key from Claude, Gemini, or GPT. your billing, your control; the key stays in your journal.
• local — a model runs right in your journal, so your data never leaves. the lane tells you whether this computer can run one, and flags when it's short on memory or missing a supported GPU before anything loads.

if a local model is selected and your machine is short on memory, switch the heavy thinking back to an off-machine lane (scout or your own key) there. transcription will also fall back to off-machine on its own when memory is tight and a cloud key is configured.

if solstone is using too much memory

1. open the thinking app and check whether the local lane is active (a setup script or pasted instructions may have switched it on). that's the usual cause of a large footprint.
2. switch back to an off-machine lane — scout or your own BYO cloud key — or confirm your machine clears the ~13 GB free-memory bar for the local model.
3. make sure you're on a current version — recent releases added the memory checks above. see keeping solstone up to date.
4. still stuck? see getting help with solstone, or file a request and include your journal doctor output.