Open any GGUF repo for an image or video model and you get a wall of files: Q8_0, Q6_K, Q5_K_M, Q4_K_S, Q3_K_M, Q2_K. They are all the same model. The only difference is how many bits each weight is stored with, and that decides two things: how big the file is, and how close the pictures stay to the original.
The short version
- Q8_0 — about half the size of the 16-bit original. In practice the pictures are the same. If it fits, take it.
- Q6_K — smaller again, still very close. A good choice when Q8 misses by a gigabyte.
- Q5_K_M / Q5_K_S — small differences in fine detail. Most people would not notice without a side-by-side.
- Q4_K_M / Q4_K_S — the popular "fits on 8–12 GB" level. Fine detail and small text start to suffer.
- Q3 and Q2 — last resort. They run, but you will see it.
A real example with FLUX.1 [dev]: the original is 23.8 GB, Q8_0 is 12.7 GB, Q5_K_S is 8.3 GB, Q4_K_S is 6.8 GB and Q2_K is 4.0 GB. Same model, six times smaller at the bottom end.
What the letters mean
The number is roughly the bits per weight. _0 and _1 are the older, simple formats. _K formats are newer and spend bits more cleverly, so a Q4_K usually looks better than a Q4_0 of almost the same size. The last letter is the mix: _S small, _M medium, _L large. M keeps a few sensitive layers at higher precision, so it is slightly bigger and slightly better than S.
A simple rule for picking
- Take your VRAM in GB.
- Take away about 0.8 GB for Windows and the driver.
- Take away the model's working memory: roughly 1–2.5 GB for image models at about one megapixel, 3–6 GB for video models on a short clip.
- Download the biggest file that fits in what is left.
That is exactly the rule this site uses to say what fits entirely. See it applied to your own GPU in the checker.
Is GGUF slower?
A little. GGUF weights are unpacked on the fly at each step, which costs some speed compared with a 16-bit or FP8 file.
For years the rule was simple: a GGUF that fits entirely in VRAM beats a bigger file that spills into system RAM. That still holds on older ComfyUI versions, and on AMD and Intel. On NVIDIA it has changed. Since March 2026 ComfyUI's Dynamic VRAM streams the part of a model that does not fit, and ComfyUI now recommends its native formats (FP8, INT8) over GGUF, saying they "will be faster even if they are larger than your memory". So on an up-to-date NVIDIA setup, try the native 8-bit file first, and fall back to the biggest GGUF that fits if it is too slow for you. I am measuring exactly this trade-off on my RTX 5060 Ti; the results will appear on the model pages.
What you need in ComfyUI
GGUF files load through the ComfyUI-GGUF custom node pack by city96: use Unet Loader (GGUF) instead of the normal diffusion model loader. The same pack has GGUF loaders for text encoders. A few uploads need their uploader's own loader (HunyuanImage 2.1 from calcuis, Ideogram 4 from molbal) — the repo page says so.