Pick your card, pick a model, get the verdict. Estimates from measured data (Apple M5 Max 128GB baseline, October 2026; NVIDIA figures scale by memory bandwidth). “Offload” = weights spill to system RAM: speed collapses, TTFT jumps 10–40x.
qwen3-coder:30b — needs 18GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
23–39 |
| RTX 3060/4070/5070 |
offload |
34–58 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
45–78 |
| RTX 3090/4090 |
fits |
146 |
| RTX 5090 |
fits |
146 |
| M5 Max (MLX) |
fits |
146 |
laguna-xs-2.1:latest — needs 18GB (공개 기록)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
20–34 |
| RTX 3060/4070/5070 |
offload |
29–50 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
39–67 |
| RTX 3090/4090 |
fits |
126 |
| RTX 5090 |
fits |
126 |
| M5 Max (MLX) |
fits |
126 |
nemotron-3.5-lightning:30b-a3b — needs 25GB (공개 기록)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
13–22 |
| RTX 3060/4070/5070 |
offload |
19–33 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
26–44 |
| RTX 3090/4090 |
offload |
39–66 |
| RTX 5090 |
fits |
115 |
| M5 Max (MLX) |
fits |
115 |
gpt-oss:20b — needs 13GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
22–38 |
| RTX 3060/4070/5070 |
offload |
33–57 |
| RTX 5060Ti-16/5070Ti/5080 |
fits |
103 |
| RTX 3090/4090 |
fits |
103 |
| RTX 5090 |
fits |
103 |
| M5 Max (MLX) |
fits |
103 |
llama3.1:8b — needs 4.9GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
fits |
101 |
| RTX 3060/4070/5070 |
fits |
101 |
| RTX 5060Ti-16/5070Ti/5080 |
fits |
101 |
| RTX 3090/4090 |
fits |
101 |
| RTX 5090 |
fits |
101 |
| M5 Max (MLX) |
fits |
101 |
qwen3.6:35b-a3b — needs 23GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
12–20 |
| RTX 3060/4070/5070 |
offload |
17–30 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
23–40 |
| RTX 3090/4090 |
tight |
86 |
| RTX 5090 |
fits |
95 |
| M5 Max (MLX) |
fits |
95 |
ornith-1.5:35b — needs 37GB (공개 기록)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
7–12 |
| RTX 3060/4070/5070 |
offload |
11–18 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
14–24 |
| RTX 3090/4090 |
offload |
21–36 |
| RTX 5090 |
offload |
28–48 |
| M5 Max (MLX) |
fits |
93 |
gemma4:26b MLX — needs 52GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
5–8 |
| RTX 3060/4070/5070 |
offload |
7–12 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
9–16 |
| RTX 3090/4090 |
offload |
14–24 |
| RTX 5090 |
offload |
19–33 |
| M5 Max (MLX) |
fits |
88 |
deepseek-r1:8b — needs 5.2GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
fits |
80 |
| RTX 3060/4070/5070 |
fits |
80 |
| RTX 5060Ti-16/5070Ti/5080 |
fits |
80 |
| RTX 3090/4090 |
fits |
80 |
| RTX 5090 |
fits |
80 |
| M5 Max (MLX) |
fits |
80 |
qwen3.8:27b — needs 17.7GB (공개 기록)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
12–20 |
| RTX 3060/4070/5070 |
offload |
17–30 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
23–40 |
| RTX 3090/4090 |
fits |
73 |
| RTX 5090 |
fits |
73 |
| M5 Max (MLX) |
fits |
73 |
qwen3.5:9b — needs 6.6GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
tight |
66 |
| RTX 3060/4070/5070 |
fits |
73 |
| RTX 5060Ti-16/5070Ti/5080 |
fits |
73 |
| RTX 3090/4090 |
fits |
73 |
| RTX 5090 |
fits |
73 |
| M5 Max (MLX) |
fits |
73 |
qwen3.8-flash-next:125b-mlx — needs 128.5GB (공개 기록)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
3–5 |
| RTX 3060/4070/5070 |
offload |
3–5 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
3–5 |
| RTX 3090/4090 |
offload |
4–7 |
| RTX 5090 |
offload |
5–9 |
| M5 Max (MLX) |
offload |
21–35 |
phi4:14b — needs 9.1GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
14–24 |
| RTX 3060/4070/5070 |
fits |
46 |
| RTX 5060Ti-16/5070Ti/5080 |
fits |
46 |
| RTX 3090/4090 |
fits |
46 |
| RTX 5090 |
fits |
46 |
| M5 Max (MLX) |
fits |
46 |
mistral-small:24b — needs 14GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
6–10 |
| RTX 3060/4070/5070 |
offload |
8–14 |
| RTX 5060Ti-16/5070Ti/5080 |
fits |
28 |
| RTX 3090/4090 |
fits |
28 |
| RTX 5090 |
fits |
28 |
| M5 Max (MLX) |
fits |
28 |
muse-glimmer:30b-mlx — needs 18GB (공개 기록)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
4–7 |
| RTX 3060/4070/5070 |
offload |
6–11 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
8–14 |
| RTX 3090/4090 |
fits |
27 |
| RTX 5090 |
fits |
27 |
| M5 Max (MLX) |
fits |
27 |
deepseek-r1:70b — needs 43GB (실측)
| Card |
Verdict |
Est. tok/s |
| RTX 5060/4060Ti-8 |
offload |
1–1 |
| RTX 3060/4070/5070 |
offload |
1–2 |
| RTX 5060Ti-16/5070Ti/5080 |
offload |
1–2 |
| RTX 3090/4090 |
offload |
2–3 |
| RTX 5090 |
offload |
3–5 |
| M5 Max (MLX) |
fits |
10 |
How to read this
- Fits — weights + 8K context live in VRAM. Expect the listed tok/s.
- Tight — fits with reduced context (~4K). ~10% slower.
- Offload — partial GPU offload; the range shown is the honest spread. Test with small context first.
Measured baseline table: 16GB measured models · M5 Max full table · cheapest VRAM per dollar
