Can I Run Qwen 3.5 on NVIDIA L40S?

Name: LLM Configurator — GPU VRAM Checker
Author: LLM Configurator

Autor: Jakub Rusinowski · Ostatnia aktualizacja: 21 lipca 2026

Yes, comfortably — you'll have ~26.9 GB of headroom running Qwen 3.5 35B-A3B at Q4_K_M (21.13125 GB, ~176 tok/s (est.)) with room for up to 64K context.

Buy This HardwareNVIDIA L40S 48GB — 48 GB VRAM · 350 W board power Deploy in the Cloud NowNVIDIA L40S on RunPod — from $0.86/hr · rate checked 2026-08

or compare on Vast.ai

As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.

Ujawnienie afiliacyjne: Niektóre odnośniki na tej stronie to linki afiliacyjne — jeśli dokonasz zakupu za ich pośrednictwem, LLM Configurator może otrzymać prowizję bez dodatkowych kosztów dla Ciebie. Jako uczestnik programu Amazon Associates, LLM Configurator zarabia na kwalifikujących się zakupach.

NVIDIA L40S 48GB

48 GB VRAM · 350 W board power

Ceny w 2026 są niestabilne — sprawdź aktualną ofertę.

Sprawdź cenę na Amazon

NVIDIA L40S Specs

VRAM	48 GB
Memory Bandwidth	864 GB/s

Qwen 3.5 35B-A3B on the NVIDIA L40S: VRAM by quantization

Quant	VRAM needed	Fits 48 GB?	Max context
F16	71.7 GB	✗ No	—
Q8_0	38.9 GB	✓ Yes	32K
Q6_K	30.4 GB	✓ Yes	64K
Q5_K_M	26.5 GB	✓ Yes	64K
Q4_K_M	22.9 GB	✓ Yes	64K
Q3_K_M	16.7 GB	✓ Yes	128K
Q2_K	13.2 GB	✓ Yes	128K

VRAM needed assumes a 4K-token context with an f16 KV cache; “Max context” is the largest window that still fits in 48 GB. Figures are estimates from parameter count, quantization and memory bandwidth — the analyzer lets you tune KV-cache quant and context.

Qwen 3.5 Sizes That Fit the NVIDIA L40S

Qwen 3.5 35B-A3B	Q4_K_M · 21.13125 GB · ~176 tok/s (est.)
Qwen 3.5 27B	Q4_K_M · 16.30125 GB · ~42 tok/s (est.)
Qwen 3.5 9B	Q4_K_M · 5.43375 GB · ~102 tok/s (est.)
Qwen 3.5 4B	Q4_K_M · 2.415 GB · ~172 tok/s (est.)
Qwen 3.5 2B	Q4_K_M · 1.2075 GB · ~237 tok/s (est.)
Qwen 3.5 0.8B	Q4_K_M · 0.48300000000000004 GB · ~314 tok/s (est.)

Buy vs. rent Qwen 3.5

Buy the GPU

~$7,499

NVIDIA L40S · MSRP

Rent by the hour

from $0.34/hr

RTX 4090 (24 GB) class

At 2 hrs/day, buying (~$7,499) beats renting at $0.34/hr after about 30.6 years.

Affiliate links — we may earn a commission if you sign up, at no extra cost to you.

RunPod $0.34/hr

Rent on RunPod →

Vast.ai $0.35/hr · typical low · varies

Rent on Vast.ai →

Cloud rates verified 2026-07 — estimates, and marketplace prices vary. Buying price is GPU MSRP only, not a full PC.

FAQ

Can I run Qwen 3.5 on the NVIDIA L40S?

Yes, comfortably — you'll have ~26.9 GB of headroom running Qwen 3.5 35B-A3B at Q4_K_M (21.13125 GB, ~176 tok/s (est.)) with room for up to 64K context.

What's the best Qwen 3.5 size for the NVIDIA L40S?

Qwen 3.5 35B-A3B at Q4_K_M quantization (21.13125 GB), estimated ~176 tokens/sec, up to 64K context.

Every Qwen 3.5 size on the NVIDIA L40S

Size	VRAM	Verdict	Speed
Qwen 3.5 397B-A17B	134.0 GB	Does not fit	—
Qwen 3.5 122B-A10B	73.7 GB	Runs at Q2_K	—
Qwen 3.5 35B-A3B	21.1 GB	Runs	~176 tok/s
Qwen 3.5 27B	16.3 GB	Runs	~42 tok/s
Qwen 3.5 9B	5.4 GB	Runs	~102 tok/s
Qwen 3.5 4B	2.4 GB	Runs	~172 tok/s
Qwen 3.5 2B	1.2 GB	Runs	~237 tok/s
Qwen 3.5 0.8B	0.5 GB	Runs	~314 tok/s

Qwen 3.5 on Other GPUs

Popular Models on the NVIDIA L40S

VRAM Tier

Best LLMs for 48 GB VRAM

Troubleshooting

Which GGUF quant should I download? (Q4 vs Q5 vs Q8)

Buying Guide

Best GPU Buyer Guide 2026

← Can I Run It? | Qwen 3.5 Model Page | NVIDIA L40S GPU Page | Check Your Hardware