FineWeb — LLM Pretraining Dataset
The gold standard for English web data. Massive, cleaned, and deduplicated dataset derived from CommonCrawl. Used to train some of the best open models including Llama 3.
Dataset Details
| Provider | HuggingFaceFW |
| Category | Pretraining |
| Size | 15 TB |
| License | ODC-By 1.0 |
| Downloads | 2.5M |
| Tags | Web, English, Cleaned, CommonCrawl |
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/fineweb")
Related datasets
- FineWeb 2 — Pretraining or continued pretraining in languages other than English
- Cosmopedia — Synthetic-textbook pretraining for small models (the SmolLM recipe)
- SlimPajama — Efficient English pretraining on a heavily deduplicated corpus
Frequently asked questions
Can I use FineWeb commercially?
Yes — FineWeb is released under ODC-By 1.0, a permissive license that allows commercial use, including training models you ship in a product. Check the dataset card for attribution requirements before release.
How much data does FineWeb contain, and do I need all of it?
FineWeb contains 15 TB. You rarely need all of it: for style and format fine-tuning, a few hundred to a few thousand examples are enough — load a slice (e.g. split="train[:1000]") and scale up only if quality plateaus.
What is FineWeb best used for?
Pretraining or continued pretraining on high-quality English web text. It belongs to the Pretraining section of our dataset hub, where you'll find alternatives and complementary sets.
← All datasets | Fine-Tuning Guide