The gold standard for English web data. Massive, cleaned, and deduplicated dataset derived from CommonCrawl. Used to train some of the best open models including Llama 3.
Dataset Details
Provider
HuggingFaceFW
Category
Pretraining
Size
15 TB
License
ODC-By 1.0
Downloads
2.5M
Tags
Web, English, Cleaned, CommonCrawl
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/fineweb")
Related datasets
FineWeb 2 — Pretraining or continued pretraining in languages other than English
Cosmopedia — Synthetic-textbook pretraining for small models (the SmolLM recipe)
SlimPajama — Efficient English pretraining on a heavily deduplicated corpus
Frequently asked questions
Can I use FineWeb commercially?
Yes — FineWeb is released under ODC-By 1.0, a permissive license that allows commercial use, including training models you ship in a product. Check the dataset card for attribution requirements before release.
How much data does FineWeb contain, and do I need all of it?
FineWeb contains 15 TB. You rarely need all of it: for style and format fine-tuning, a few hundred to a few thousand examples are enough — load a slice (e.g. split="train[:1000]") and scale up only if quality plateaus.
What is FineWeb best used for?
Pretraining or continued pretraining on high-quality English web text. It belongs to the Pretraining section of our dataset hub, where you'll find alternatives and complementary sets.