What Fits on the Graphics Card You Already Own

One multiplication answers the whole question. Four-bit weights need roughly 0.57GB per billion parameters, so an 8B model is about 4.6GB and a 32B model about 18.2GB, before context.

Work from the card you own rather than from a buyer’s guide and most people discover they already have what they need.

The ladder

Your card	Usable after headroom	Largest 4-bit model that fits
8 GB	~6 GB	8B
12 GB	~10 GB	14B
16 GB	~14 GB	14B
24 GB	~22 GB	32B
32 GB	~30 GB	32B

Arithmetic, not benchmarks. Two gigabytes are subtracted from each card for context and the desktop, which is generous for a short context and tight for a long one.

The two rows that matter are the ones that repeat

Look at the table again. 16GB and 32GB hold the same class of model, and so do 12GB and 16GB. The steps that look biggest on a spec sheet buy headroom rather than capability.

The one upgrade that unlocks a genuinely different size is 16GB to 24GB, which is the difference between a 14B model and a 32B one. That is a real change in what the thing can do; the others are a change in how comfortably it does the same thing.

If you are spending money on this, spend it on crossing that line rather than on the biggest number available.

Where the arithmetic stops helping

It says nothing about speed. A model that fits generates at a rate governed mostly by memory bandwidth, which differs by a factor of two or more between cards holding the same amount. Fitting is a yes-or-no test; speed is a separate question with a separate answer.

Context is not free. A long conversation occupies the same memory the weights do, and a big context window can consume more than the model itself, which is the mechanism behind why chatbots forget.

Four-bit is a compromise. It is the sensible default and it does cost some quality. At eight-bit every figure here roughly doubles, which moves a 14B model from about 8GB to about 14GB and off most cards.

When no card is the answer

At 70B the weights alone are about 40GB at four-bit, and no consumer graphics card holds that. That is the point where the question stops being which card and becomes which machine, and where a unified-memory desktop changes the arithmetic entirely.

For nearly everyone reading this, the 70B row is a curiosity rather than a plan.

What to actually do

• Look up your card’s memory before buying anything. The number is the whole answer and most people have never checked it.
• Start with a model two sizes below your ceiling. It leaves room for a long context, and the difference between an 8B and a 14B model is smaller than the difference between a model that fits and one that swaps.
• Upgrade to cross 24GB, or do not upgrade. Every other step buys headroom.
• Add system memory before replacing the card. Partial offload is slow and it is much cheaper than a new card, and it turns a model that will not load into one that will.