How Much VRAM You Need to Run an LLM on Your Own Machine

Start with the model’s download size, then assume you need more. The file has to fit in memory, and the working memory the model needs on top of it grows with how much text you ask it to hold.

The good news is that you do not have to estimate the first part. Every model in Ollama’s library publishes its size next to its name, so the floor is a fact you can look up rather than a calculation you can get wrong.

The published sizes, which are the useful anchor

Google’s Gemma 3 comes in five sizes, and Ollama lists each one:

Model	Download size
gemma3:270m	292 MB
gemma3:1b	815 MB
gemma3:4b	3.3 GB
gemma3:12b	8.1 GB
gemma3:27b	17 GB

Read down that column and the shape of the whole problem appears. Parameter counts and memory scale together, roughly two-thirds of a gigabyte per billion parameters at the default precision — so an 8 GB card is a 12B-class machine, and a 27B model needs a 24 GB card to sit comfortably.

Quantisation is why the same model has two sizes

The default tags above are not the raw model. Ollama’s library lists gemma3:1b at 815 MB with digest 8648f39daa8f — and lists gemma3:1b-it-q4_K_M with the same digest and the same 815 MB. The default download is the four-bit version.

The full-precision tag of that identical model, gemma3:1b-it-fp16, is 2.0 GB.

Same model, same capabilities on paper, two and a half times the memory. Quantisation stores each weight in fewer bits, and it is the single reason consumer hardware can run these models at all. When someone says a 27B model runs on a 24 GB card, they mean the quantised one — nobody is loading it at full precision.

Context costs memory too, and this is what people forget

The model file is the floor. The conversation is the rest.

Ollama’s own documentation is blunt about it: “Setting a larger context length will increase the amount of memory required to run a model.” And the defaults it ships are set by how much VRAM it finds:

• Under 24 GiB VRAM — 4k context
• 24–48 GiB — 32k context
• 48 GiB or more — 256k context

That first tier is the one most people are in, and 4k tokens is a short conversation. The same documentation notes that tasks “which require large context like web search, agents, and coding tools should be set to at least 64000 tokens” — sixteen times the default you were given. Turning that up is exactly the moment a model that fit stops fitting.

It compounds if you run more than one request at once. Ollama’s FAQ states that required RAM “will scale by OLLAMA_NUM_PARALLEL * OLLAMA_CONTEXT_LENGTH”, and that a “2K context with 4 parallel requests will result in an 8K context and additional memory allocation.”

The check that tells you whether it actually fit

This is the part worth learning, because it replaces guessing with a reading. Ollama’s documentation says to “avoid offloading the model to CPU” and to “verify the split under PROCESSOR” by running:

ollama ps

The output names the model, its size in memory, and the split:

NAME             ID              SIZE      PROCESSOR    CONTEXT    UNTIL
gemma4:latest    c6eb396dbd59    9.6 GB    100% GPU     131072     2 minutes from now

100% GPU means it fit. Anything else — 40% GPU / 60% CPU — means part of the model is living in system RAM, and that part is being computed by a processor that is far slower at this than your graphics card. It will still answer. It will answer slowly, and now you know why.

Note the SIZE column too: 9.6 GB in memory for a model whose download is smaller. The gap is the context.

What people actually live with

The documented limits and the reported experience agree, which is reassuring. In a large r/LocalLLaMA thread asking exactly this, one person on 12 GB says they “generally use the quantized version of Gemma 12B” — which is precisely what the table above predicts for that card.

Another on the same 12 GB describes the trade-off honestly: “I also mainly stuck to the 8-12b q4 models, but lately I’ve found that I can also live with the 5tok/s from gemma3 27B if I just need 3-4 answers”. That is the spill to CPU, quantified. Five tokens a second is unpleasant for chat and perfectly fine for a job you leave running.

And at the far end, someone with a 512 GB Mac reports “about 16-18 t/s on these models” and adds the sentence that should temper anyone’s upgrade plans: “It doesn’t run any models particularly fast, it’s more just that you can at all.”

Apple Silicon changes the arithmetic

On a Mac, there is no separate pool of video memory — the GPU draws on the same unified memory as everything else. A 32 GB MacBook is therefore closer to a 32 GB graphics card than to a 32 GB PC with an 8 GB GPU, and the thread has people running on an M3 Max what others run on dedicated cards.

The trade is speed rather than capacity, which is why the same thread has Mac owners quoting sixteen tokens a second where a stack of GPUs would be far quicker.

What to actually do

• Look up the file size before you download anything. It is on the model’s tags page. If it is larger than your VRAM, you already know the answer.
• Leave headroom of a few gigabytes. The file size is not the running size, and ollama ps will show you the difference on your own machine.
• Run ollama ps after the first prompt. If PROCESSOR does not say 100% GPU, drop to the next size down before concluding the model is bad.
• Match the card to the class you want, not the biggest number you have seen. 8 GB comfortably runs models up to about 12B; 12 GB is the same class with room for context; 24 GB is where 27B-class models stop being a compromise.
• If you are buying a Mac for this, buy memory. It is the one spec that decides what will run, and it cannot be added later.