NVIDIA Cosmos 3: The Open World Model That Teaches Robots to Think Before They Act

Most of the AI you read about writes text. NVIDIA Cosmos 3, launched on June 1, 2026, does something different: it learns how the physical world behaves — how objects move, fall, collide and respond — and uses that understanding to imagine what happens next and to decide what a robot or car should do. NVIDIA calls this category Physical AI, and Cosmos 3 is its open, frontier-scale foundation model.

It's also the first fully open "omnimodel" that natively takes in and generates text, images, video, ambient sound and actions in a single system. Here's what that actually means, why the architecture is clever, what it takes to run, and the strategy behind giving it away.

What is a "world model," and why does robotics need one?

A large language model learns the statistics of language. A world model learns the statistics of reality: given what a camera sees right now, what is likely to happen a moment later? Drop a mug and it falls and shatters; push a box and it slides; a pedestrian steps off a curb and a car must react. That intuition — physical common sense — is exactly what robots and autonomous vehicles have always lacked.

The reason this matters is data. Teaching a robot arm or a self-driving stack by trial-and-error in the real world is slow, expensive and sometimes dangerous. A good world model lets you do it in simulation instead: generate endless photorealistic, physically plausible video of scenarios, predict how actions play out, and evaluate a policy — all in software. NVIDIA's pitch is blunt: Cosmos 3 compresses physical-AI training and evaluation cycles from months to days.

The clever part: two towers that think before they act

Under the hood, Cosmos 3 is a two-tower Mixture-of-Transformers (MoT). Instead of one monolithic network, it splits the job across two specialized "towers" that share attention but keep their own normalization layers and MLPs:

• The reasoner tower is an autoregressive vision-language model — the "brain." It reads the images, video and text it's given and reasons about motion, object interactions and physical cause-and-effect. This is the thinking step.
• The generator tower is diffusion-based. Conditioned on the reasoner's understanding, it generates physics-aware future video and the corresponding action sequences. This is the acting step.

Splitting understanding from generation is what lets Cosmos "reason before it renders," and it's why NVIDIA frames the model around the phrase "think before it acts." If the Mixture-of-Transformers idea sounds familiar, it's a cousin of the Mixture-of-Experts sparsity behind open LLMs like DeepSeek V4 and Thinking Machines' Inkling — but here the "experts" are whole specialized towers for perception vs. prediction.

One family, three sizes

Cosmos 3 isn't a single model — it's a family trained at three scales, so you can pick the trade-off between fidelity and where it runs. Each is a total parameter count built on a smaller dense transformer backbone:

• Cosmos 3 Super (64B) is the high-fidelity model you post-train: take the open base, fine-tune it on your fleet's data, and build a bespoke robotics or self-driving world model. NVIDIA reports Super takes the top overall PAI-Bench-G scores on both text-to-video and image-to-video across every model tested, including closed ones.
• Cosmos 3 Nano (16B) is the balance point — high-quality video and action reasoning in a fraction of a second, cheap enough to run in a serious workstation.
• Cosmos 3 Edge (4B), added in mid-July 2026, is the one hobbyists will actually touch: small enough to run reasoning and robot-action generation on the robot itself, with no cloud round-trip. On an NVIDIA Jetson Thor it produces 32 actions per inference at a real-time 15 Hz.

Curious whether a specific card clears the bar? The Hardware Analyzer and the per-model Cosmos 3 page run the VRAM math for you.

One model, every physical modality

What makes Cosmos 3 an "omnimodel" is that all of this — perception and generation — happens across every modality that matters for embodied AI, in one network:

That training diet — roughly 20 trillion tokens spanning nearly a billion images, 400M+ real and synthetic videos, plus audio and action logs from humans and robots — is what gives Cosmos its physical grounding. It's a very different recipe from a text LLM's, and it's why the model can reason about a dropped object or a merging car rather than just describe one.

How good is it?

Cosmos 3 is designed to top the open leaderboards for physical AI, and NVIDIA reports it does — while introducing its own evaluation framework, HUE (Cosmos Human Evaluation), which breaks each generated video into yes/no fact questions across dimensions like semantic alignment, physical-law consistency, geometric reasoning and visual integrity.

Two honest caveats. These are vendor-reported launch numbers — independent replication of world-model benchmarks takes time, so read them as "reported," not yet confirmed. And "state-of-the-art open model" is the specific claim; several closed systems still compete hard on individual axes. On this site, Cosmos 3 is flagged preview for exactly that reason.

Can you actually run it?

For once, part of this family is genuinely within reach. Super is a datacenter animal (~128 GB of VRAM at BF16 → an H100/H200/B200-class node), and Nano wants a 96 GB workstation card like the RTX PRO 6000. But Edge is a 4B model with an ~8 GB footprint that NVIDIA explicitly positions for a GeForce RTX 3070 or better as a prototyping on-ramp, and for Jetson modules on the robot itself.

The weights are open — distributed under the permissive OpenMDW-1.1 license (the Linux Foundation's Open Model Development Weights license, which permits commercial use and derivatives) — and available on Hugging Face under the nvidia/ organization and on NVIDIA NGC. If you're weighing on-device inference versus a cloud pipeline for a robotics project, our local vs cloud AI comparison covers the trade-offs, and the Hardware Analyzer will tell you which tier your GPU can host.

Why give it away? The open bet — and the Cosmos Coalition

Handing out a frontier world model for free is a strategy, not charity. NVIDIA's business is selling the GPUs that train and run physical AI; the faster the whole robotics and autonomous-vehicle ecosystem moves, the more compute it needs. An open, best-in-class base model that everyone post-trains is the fastest way to make that happen — and it keeps NVIDIA's stack at the center of it.

To push that further, NVIDIA launched the Cosmos Coalition, a group of world-model builders and AI developers — including Agile Robots, Black Forest Labs, Generalist, LTX, Runway and Skild AI — collaborating on the next generation of world models. It's the same open-weights logic reshaping the LLM world, aimed squarely at embodied AI.

The bottom line

Cosmos 3 is the clearest sign yet that the "foundation model" era is expanding past text and into the physical world. It's an open, omnimodal, physics-grounded model that reasons about a scene and predicts what happens next — the missing engine that lets robots and self-driving systems learn in simulation instead of the hard, slow, dangerous real world.

You won't be chatting with it. But if the last few years were about models that understand language, Cosmos 3 is a bet that the next few are about models that understand reality — and NVIDIA just handed the blueprint to everyone.

---

Explore all three sizes and the full specs on the Cosmos 3 model page → Related open frontier models: Thinking Machines' Inkling → · DeepSeek V4 Pro → Browse the full Model Library → or check your rig with the Hardware Analyzer →