Zhipu AI (Z.ai)744B~450 GB VRAM at Q4_K_M

GLM-5 744B — VRAM, speed & local setup

Written by Jakub Rusinowski · Last updated February 15, 2026

Frontier MoE model with 744B total and 40B active parameters. Scores 77.8% on SWE-bench Verified and holds the highest open-source Chatbot Arena Elo at 1451. Trained on Huawei Ascend hardware — no NVIDIA dependency. Requires ~400+ GB VRAM at Q4; accessible via API at ~$0.80/1M input tokens.

GLM-5 744B needs about 450 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.

Logic95

Creative91

Coding96

VRAM and speed by quantization

Quoted against NVIDIA RTX 4090 (24 GB). Weights plus overhead only: this model's architecture is unpublished, so no KV cache is added.

Quant	VRAM	Speed (est.)	Fit
Q2_K 2.63 bpw	245.4 GB	—	Too big
Q3_K_M 3.41 bpw	317.9 GB	—	Too big
Q4_K_M 4.83 bpw	450 GB	—	Too big
Q5_K_M 5.67 bpw	528.1 GB	—	Too big
Q6_K 6.56 bpw	610.9 GB	—	Too big
Q8_0 8.50 bpw	791.3 GB	—	Too big
F16 16.00 bpw	1488.8 GB	—	Too big

Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. GLM-5 744B VRAM calculator →

Get GLM-5 744B running

GLM-5 744B is not practical on a single consumer GPU: it needs about 450 GB at Q4_K_M, more than any single device in our catalogue holds with headroom. How multi-GPU inference works, or rent a multi-GPU server by the hour:

Deploy in the cloud now RunPod

or compare on Vast.ai

As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.

How to run GLM-5 744B

Install Ollama, then run:

$ollama run glm-5

Weights on Hugging Face: THUDM/GLM-5 ↗

Specifications

Preview — The model is released, but these specs are thin or rest on a single source. Individual fields may be wrong.

Parameters: 744 Billion (40B active)
Context window: 200,000
Architecture: Mixture-of-Experts + DeepSeek Sparse Attention
Provider: Zhipu AI (Z.ai)
Licence: MIT
Practical deployment: Multi-GPU server
Specified at: Q4_K_M
System RAM: 1024 GB
Record updated: 2026-02-15

LicenceMITCommercial use permitted

Commercial use permitted. No usage restrictions beyond attribution.

Quality and use cases

Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.

Best forsoftware engineeringcodingenterprisecloud api

Benchmark	Score	Provenance
SWE-bench Verified	77.8 / 100 %	reported

Can I run GLM-5 744B on my GPU?

GLM-5 744B — frequently asked questions

How much VRAM does GLM-5 744B need?

About 450 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.

Does GLM-5 744B run on an RTX 4090 (24 GB)?

No. GLM-5 744B needs about 450 GB at Q4_K_M, more than a single RTX 4090's 24 GB. It needs a larger card, several GPUs, or Apple Silicon with enough unified memory — or it runs with part of the weights offloaded to system RAM, which is much slower.

How do I run GLM-5 744B locally?

Install Ollama and run `ollama run glm-5`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.

What other sizes does GLM-5 / GLM-5.1 come in?

GLM-5 9B (6 GB), GLM-5 32B (20 GB), GLM-5.1 72B (44 GB), GLM-5 744B (450 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.