It's easy to assume that better suggestions require a bigger model — more parameters, more compute, a datacenter GPU behind an API call. For open-ended generation, that assumption often holds. But next-phrase autocomplete is a fundamentally different, narrower task, and a compact, quantized model running on a laptop CPU is well matched to it.

Autocomplete Is a Constrained Task, Not an Open One

When you ask a large language model to write an essay from a blank page, it has to make thousands of decisions about structure, argument, and tone with almost no constraints. That's where model scale pays off — more parameters generally means a broader range of plausible outputs to draw from.

Next-phrase prediction is a different problem. The model already has the sentence you've started, the surrounding context, and (in the case of Typeahead) the app you're typing in as strong signal. It isn't generating from nothing — it's completing a short, already-constrained continuation. That constraint is exactly what makes a small model viable: fewer plausible completions to choose between means less raw capacity is needed to choose well.

Analogy: predicting the next line of a sentence is closer to finishing someone else's thought than writing your own from scratch. The second is open-ended; the first has a much smaller space of reasonable answers.

What Quantization Does

A raw language model's weights are normally stored as 16- or 32-bit floating point numbers. Quantization compresses those weights to lower precision — 4-bit or 8-bit representations, for instance — with a small, generally acceptable accuracy trade-off. The result is a model file a fraction of its original size, fast enough to run useful inference on ordinary consumer hardware, including a laptop with no dedicated GPU.

This is what makes it realistic to ship a model that lives entirely on someone's Windows PC rather than a server rack: a quantized model in the few-hundred-megabyte to low-gigabyte range fits comfortably on disk and in memory alongside everything else already running.

How Typeahead Sizes Its Models

Typeahead for Windows offers three optional models, letting you choose the trade-off between speed and suggestion quality that fits your machine:

Fast (~400 MB) — default

The smallest model, tuned for low latency. This is what's active out of the box, and it's sufficient for most everyday phrase completion.

Balanced (~1.1 GB)

A step up in model capacity for machines that can spare the extra memory and disk, trading a bit of speed for broader suggestion range.

Maximum (~2.5 GB)

The largest of the three options, for users who want the best suggestion quality Typeahead offers and have the hardware headroom for it.

All three run through llama.cpp, an inference engine built specifically for running language models efficiently on ordinary CPUs and consumer GPUs, and all three are built on Qwen 3, a multilingual model family that covers 100+ languages — so the suggestion language follows whatever you're actively typing in, without a manual language switch.

Why This Also Means No Cloud Dependency

Because the model is small enough to run locally, Typeahead doesn't need a server to do the heavy lifting. That has a direct privacy consequence — nothing you type is transmitted anywhere — but it also has a practical one: suggestions work with no internet connection, no API rate limits, and no dependency on a remote service staying online.

Model Approx. size Trade-off
Fast ~400 MB Lowest latency, default choice
Balanced ~1.1 GB More capacity, still lightweight
Maximum ~2.5 GB Best suggestion quality available

What You Get Before Downloading a Model at All

Typeahead doesn't require a model to provide value on day one. Custom snippet autocorrect and dictionary-based autocomplete work immediately after installation, before any model has been downloaded — the AI-driven sentence suggestions layer on top once you choose one of the three models.

Frequently Asked Questions

Is a 400 MB model really good enough for useful suggestions?
For next-phrase prediction — completing a sentence you've already started — yes. It's a narrower task than open-ended writing, and the fast model is Typeahead's default for exactly that reason.
What does quantization mean, in plain terms?
It's a technique that compresses a model's internal numbers to lower precision, shrinking the file size and speeding up inference, with a small trade-off in accuracy that's generally acceptable for this kind of task.
Which model should I choose?
Start with the fast (~400 MB) default. If your machine has memory and disk to spare and you want broader suggestion quality, try balanced or maximum — you can switch at any time.
Do I need a model downloaded to use Typeahead at all?
No. Custom snippet autocorrect and dictionary-based autocomplete work before any model is downloaded. The AI sentence-completion feature becomes available once you pick a model.
What inference engine does Typeahead use?
llama.cpp, running the Qwen 3 model family, chosen specifically for efficient inference on ordinary consumer CPUs and GPUs without requiring specialized hardware.

Get Typeahead

Choose fast, balanced, or maximum. All included, no upcharge. One-time $9.99.

 Get it on Microsoft Store