"It runs a language model on your machine" is a sentence that makes people picture a fan spinning up and a laptop that gets warm doing nothing. That is a reasonable thing to picture, because it describes what happens when you run a chat model badly.
Typing suggestions are a different workload. Here is what they actually cost, so you can size them for the machine you have.
Four costs, and only one is constant
Disk. The model file, once. A few hundred megabytes to a few gigabytes, paid at download.
Memory. The weights have to stay resident while the model is loaded: roughly the size of the file, plus a modest working overhead. This is the constant cost and the one that decides whether a size fits your machine. On a 16 GB laptop, one gigabyte of resident model is unremarkable. On an 8 GB laptop with a browser open, three gigabytes is not.
Compute. Only while a suggestion is being produced. For sentence completion that is a burst of a few hundred milliseconds, a handful of times a minute, dominated by the pauses between your own keystrokes. Compare that with a chat assistant generating six hundred words, which runs the processor flat out for several seconds.
Battery. Follows compute. Short bursts cost little; the honest caveat is that a machine which never idles into its deep power states costs more than one that does, so the effect is not exactly zero.
The number to hold on to: typing prediction is idle almost all of the time. Resident memory is the real cost, and everything else is noise unless the model is badly oversized for the machine.
Why the file is smaller than you expect
A model advertised at a few hundred megabytes is not a few hundred million parameters. Models are distributed quantised, with the weights stored at reduced precision — commonly four or five bits instead of sixteen. That is roughly a fourfold reduction in size and memory for a modest, measurable loss of quality.
For open-ended reasoning that loss matters. For finishing a sentence you have already started it mostly does not, because the constraints come from your own text rather than from the model's depth. That is the technical reason a small local model is viable for this task and not for others.
Latency is the real limit, not quality
There is a hard deadline. A suggestion that arrives after you have typed the words yourself is worse than no suggestion: it appears just as you finish, and you read it for nothing.
That deadline caps model size regardless of how much memory you have. Past a certain point a better model is a slower model, and slower means unused. The practical target is a suggestion that appears while your fingers are still in the gap between words — on the order of a few hundred milliseconds, not a second and a half.
This inverts the usual intuition. Bigger is not better here past the point where it is late. The right size is the largest that still answers inside the gap on your hardware, which is a different answer on a 2019 ultrabook and on a desktop with a discrete graphics card.
Choosing a size
Typeahead ships three, which map onto the trade-off directly.
| Size | On disk | Stated requirement | Who it is for |
|---|---|---|---|
| Fast (default) | ~600 MB | — | Any supported machine, and the right place to start |
| Balanced | ~1.2 GB | 8 GB system memory | Machines with headroom: better suggestions, still comfortably fast |
| Maximum quality | ~2.5 GB | Discrete graphics card with 8 GB of its own memory, 16 GB system memory | Desktops and workstations with a real GPU |
Two things about that table.
Start at Fast even if your machine can take more. Use it for a week. If the suggestions are usefully accurate, a larger model buys you nothing you will notice while typing a chat message. If they are noticeably off, move up — and you now know what you are trading latency for.
The maximum tier wants a discrete card with its own 8 GB, not 8 GB of shared system memory. Integrated graphics on a laptop do not satisfy that, and forcing a model onto hardware that has to page it is exactly how you produce the fan-spinning experience this article opened with.
Machines that are genuinely tight
- 8 GB of RAM with a browser, Teams and Office open. You are already close to the limit before anything else loads. Use the smallest model and expect to notice it more than someone with 16 GB. If your machine already swaps during a normal day, a resident model makes that worse.
- Older processors without AVX2. Inference libraries lean on vector instructions. A pre-2013 processor will be slower than any quoted figure, and the latency ceiling above is what bites.
- Thermally limited thin laptops. Bursts are short enough that sustained throttling is unlikely from this workload alone, but a machine already throttling under other load has nothing spare.
- On battery, away from power. Suggestions still work, because they are local and need no connection. If you are optimising for a long flight, the smaller model is the one to be on.
How to check it on your own machine
Numbers measured on someone else's laptop tell you very little. Two measurements, ten minutes.
- 1. Before installing anything, open Task Manager → Performance and note your memory in use during a normal working setup: your usual browser tabs, mail, chat.
- 2. After installing and loading a model, look at the same figure, and at the app's own memory in the Processes tab.
If the difference leaves you comfortably below your total, you are fine. If it puts you against the ceiling, drop a size.
Latency is subjective and better for it: type a sentence you would normally type and notice whether the suggestion is there when your fingers pause, or arrives after. A trial period is the right way to answer that — Typeahead's is seven days, with every model size unlocked.
What you get for the cost
- Nothing you type leaves the machine. Not a policy about how a server handles your text, but a property of where it is processed.
- It works with no connection. On a plane, on a train, on hotel Wi-Fi that has stopped resolving DNS.
- No per-seat subscription and no account, because there is no server to pay for.
- No variable latency. A local model is the same speed at 9am and at 5pm, which a shared service is not.
Whether that is worth a gigabyte of resident memory is a genuine question with a different answer per person. It is a smaller question than it sounds, and it is answerable in a week.