FineWeb
The gold standard for English web data. Massive, cleaned, and deduplicated dataset derived from CommonCrawl. Used to train some of the best open models including Llama 3.
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/fineweb")Preview a sample row
[
{
"id": "fw_001",
"text": "The solar system formed 4.6 billion years ago from the gravitational collapse of a giant interstellar molecular cloud.",
"url": "https://science-daily.example.com/solar-system",
"date": "2023-11-15"
}
]Frequently asked questions
Can I use FineWeb commercially?
Yes — FineWeb is released under ODC-By 1.0, a permissive license that allows commercial use, including training models you ship in a product. Check the dataset card for attribution requirements before release.
How much data does FineWeb contain, and do I need all of it?
FineWeb contains 15 TB. You rarely need all of it: for style and format fine-tuning, a few hundred to a few thousand examples are enough — load a slice (e.g. split="train[:1000]") and scale up only if quality plateaus.
What is FineWeb best used for?
Pretraining or continued pretraining on high-quality English web text. It belongs to the Pretraining section of our dataset hub, where you'll find alternatives and complementary sets.