FineWeb 2 — LLM 预训练 Dataset
The FineWeb quality-filtering methodology extended to over 1,000 languages (1,868 language-script pairs) — roughly 8TB of compressed, deduplicated web text (~3 trillion words) from 96 CommonCrawl snapshots. Per-language pipelines were tuned and validated so filtered data beats unfiltered in ablation training. The default multilingual pretraining corpus for open models.
Dataset Details
| Provider | HuggingFaceFW |
| Category | 预训练 |
| Size | 8TB Compressed |
| License | ODC-By 1.0 |
| Downloads | n/a |
| Tags | Multilingual, CommonCrawl, Deduplicated, 1000+ Languages |
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/fineweb-2")
相关数据集
- FineWeb — Pretraining or continued pretraining on high-quality English web text
- Aya Dataset — Instruction-tuning in non-English languages with real human-written data
- Cosmopedia — Synthetic-textbook pretraining for small models (the SmolLM recipe)
- SlimPajama — Efficient English pretraining on a heavily deduplicated corpus
常见问题
FineWeb 2 可以商用吗？
可以——FineWeb 2 采用 ODC-By 1.0 宽松许可证，允许商业使用，包括训练用于产品的模型。发布前请查看数据集卡片中的署名要求。
FineWeb 2 有多少数据？需要全部使用吗？
FineWeb 2 包含 8TB Compressed。通常不需要全部：风格和格式微调只需几百到几千条样本——先加载切片（如 split="train[:1000]"），质量到达瓶颈时再扩大规模。
FineWeb 2 最适合做什么？
Pretraining or continued pretraining in languages other than English。它属于数据集中心的「预训练」板块，那里有替代和互补的数据集。
← 全部数据集 | Fine-Tuning Guide