The largest open code dataset ever assembled: 900 billion tokens across 600+ programming languages sourced from GitHub, with an opt-out mechanism for developers. Powers StarCoder2 and is the go-to pretraining corpus for code LLMs. Includes permissively-licensed repos with deduplication and near-deduplication passes.
Dataset Details
Provider
BigCode
Category
Code
Size
900B Tokens
License
Various (per-file)
Downloads
350k
Tags
Source-Code, 600+ Languages, GitHub, BigCode
from datasets import load_dataset
ds = load_dataset("bigcode/the-stack-v2")
Check the terms first — The Stack v2 is distributed under "Various (per-file)", a custom or mixed license. Read the dataset card carefully before using it in any commercial product.
How much data does The Stack v2 contain, and do I need all of it?
The Stack v2 contains 900B Tokens. You rarely need all of it: for style and format fine-tuning, a few hundred to a few thousand examples are enough — load a slice (e.g. split="train[:1000]") and scale up only if quality plateaus.
What is The Stack v2 best used for?
Pretraining code models across 600+ programming languages. It belongs to the Code section of our dataset hub, where you'll find alternatives and complementary sets.