The Stack v2 — LLM Code Dataset
The largest open code dataset ever assembled: 900 billion tokens across 600+ programming languages sourced from GitHub, with an opt-out mechanism for developers. Powers StarCoder2 and is the go-to pretraining corpus for code LLMs. Includes permissively-licensed repos with deduplication and near-deduplication passes.
Dataset Details
| Provider | BigCode |
| Category | Code |
| Size | 900B Tokens |
| License | Various (per-file) |
| Downloads | 350k |
| Tags | Source-Code, 600+ Languages, GitHub, BigCode |
from datasets import load_dataset
ds = load_dataset("bigcode/the-stack-v2")
Related datasets
Frequently asked questions
Can I use The Stack v2 commercially?
Check the terms first — The Stack v2 is distributed under "Various (per-file)", a custom or mixed license. Read the dataset card carefully before using it in any commercial product.
How much data does The Stack v2 contain, and do I need all of it?
The Stack v2 contains 900B Tokens. You rarely need all of it: for style and format fine-tuning, a few hundred to a few thousand examples are enough — load a slice (e.g. split="train[:1000]") and scale up only if quality plateaus.
What is The Stack v2 best used for?
Pretraining code models across 600+ programming languages. It belongs to the Code section of our dataset hub, where you'll find alternatives and complementary sets.
← All datasets | Fine-Tuning Guide