LLM 训练数据集中心

作者： Jakub Rusinowski · 最后更新： 2026年7月21日

47 个精选开放数据集，用于 LLM 微调与训练——指令微调、推理轨迹、函数调用、偏好数据等。每个条目都包含准确的许可证、规模和 load_dataset() 代码片段。

从这里开始：一小时微调你的第一个 LLM · 微调参考指南（SFT、LoRA 与 DPO） · 检测你的 GPU 能否微调

对比全部 47 个数据集

点击列标题排序。点击数据集名称查看完整卡片，含许可证详情、样例数据和代码。

数据集	类别	规模	许可证	最适合
FineWeb	预训练	15 TB	ODC-By 1.0	Pretraining or continued pretraining on high-quality English web text
Cosmopedia	预训练	25B Tokens	Apache 2.0	Synthetic-textbook pretraining for small models (the SmolLM recipe)
Python-Edu	指令 / SFT	400 GB	MIT	Continued pretraining for Python code understanding
OpenHermes 2.5	指令 / SFT	1M Rows	Apache 2.0	The default general-purpose SFT mix for 7B-13B fine-tunes
UltraFeedback	偏好（RLHF / DPO）	64k Rows	MIT	The default DPO preference set to run after any SFT pass
LLaVA Instruct	视觉	158k Samples	CC-BY-NC 4.0	Adding image understanding to an open LLM (the LLaVA recipe)
SlimPajama	预训练	627B Tokens	Apache 2.0	Efficient English pretraining on a heavily deduplicated corpus
Databricks Dolly 15k	指令 / SFT	15k Rows	CC-BY-SA 3.0	Commercially safe instruction tuning - fully human-written
Stanford Alpaca	指令 / SFT	52k Rows	CC-BY-NC 4.0	Learning the classic instruction format; research only (CC BY-NC)
ShareGPT 52K	指令 / SFT	52k Conversations	CC-BY-NC 4.0	Teaching natural multi-turn dialogue (non-commercial)
WizardLM Evol Instruct 70k	指令 / SFT	70k Rows	Apache 2.0	Boosting complex-instruction handling with Evol-Instruct data
LIMA: Less Is More for Alignment	指令 / SFT	1k Rows	CC-BY-NC-SA 4.0	Style and format alignment with a tiny curated set - quality over quantity
Smoltalk	指令 / SFT	1M Rows	Apache 2.0	General SFT for small models (the SmolLM2 recipe)
The Stack v2	代码	900B Tokens	Various (per-file)	Pretraining code models across 600+ programming languages
Magicoder-OSS-Instruct-75K	代码	75k Rows	MIT	Code instruction tuning grounded in real open-source snippets
CodeFeedback-Filtered-Instruction	代码	74k Rows	Apache 2.0	Training coding models that respond to execution feedback
MMLU (Massive Multitask Language Understanding)	评估与基准测试	16k Questions	MIT	Benchmarking general knowledge - never train on it
HumanEval	评估与基准测试	164 Problems	MIT	Benchmarking Python code generation - never train on it
Winogrande	评估与基准测试	44k Problems	Apache 2.0	Benchmarking commonsense reasoning
MT-Bench	评估与基准测试	80 Questions	Apache 2.0	Judging multi-turn chat quality with LLM-as-judge
Anthropic HH-RLHF	偏好（RLHF / DPO）	170k Pairs	MIT	Safety-focused preference training (helpfulness and harmlessness)
DPO Mix 7K	偏好（RLHF / DPO）	7k Pairs	Apache 2.0	A small, balanced DPO starter set
MetaMathQA	推理	395K Pairs	MIT	Boosting GSM8K/MATH-style math skills in 7B models
Orca Math Word Problems	推理	200K Problems	MIT	Grade-school math word problems for small models
Open-Platypus	指令 / SFT	25K Questions	CC BY NC 4.0	A quick STEM and logic boost on a small budget (non-commercial)
OpenOrca	指令 / SFT	4.2M Samples	MIT	Explanation-style SFT at scale (the Orca recipe)
Infinity-Instruct	指令 / SFT	7.5M Instructions	Apache 2.0	Large-scale general SFT when you need millions of samples
Magpie-Align	指令 / SFT	3M Pairs	Apache 2.0	Fresh synthetic SFT data without seed data or scraping
HelpSteer2	偏好（RLHF / DPO）	21K Samples	CC BY 4.0	Training reward models with fine-grained quality ratings
Nectar	偏好（RLHF / DPO）	183K Prompts	Apache 2.0	Reward-model training with 7-way ranked responses
Tulu 3 SFT Mix	指令 / SFT	939K Samples	ODC-BY	Reproducing a state-of-the-art fully open post-training recipe
SWE-bench	评估与基准测试	2,294 Tasks	MIT	Benchmarking real-world software engineering - never train on it
MATH (Hendrycks)	评估与基准测试	12,500 Problems	MIT	Benchmarking competition math ability
NuminaMath	推理	860K Problems	Apache 2.0	Chain-of-thought math fine-tuning up to olympiad level
WebInstruct	指令 / SFT	220M Pairs	CC BY-SA 4.0	Web-scale instruction mining for domain skills
LMSYS-Chat-1M	指令 / SFT	1M Conversations	CC BY-NC 4.0	Studying real-world usage patterns across 25 models (non-commercial)
WildChat-1M	指令 / SFT	1M Chats	AI2 ImpACT	Training on real user conversations; safety and robustness research
No Robots	指令 / SFT	10k Rows	CC BY-NC 4.0	Your first fine-tune — small, clean, 100% human-written SFT data (non-commercial license)
SmolTalk 2	指令 / SFT	3 Subsets (Mid 4.8M Rows)	Apache 2.0	Reproducing a complete modern post-training pipeline (mid-training → SFT → preference) for small models
OpenThoughts3-1.2M	推理	1.2M Rows	Apache 2.0	Distilling strong math/code/science reasoning into 7B–32B models
s1K-1.1	推理	1k Rows	Apache 2.0	Cheap, fast reasoning fine-tunes — 1k samples means minutes of training, not days
Glaive Function Calling v2	智能体与函数调用	113k Rows	Apache 2.0	Teaching a model basic function-calling: when to call, what JSON to emit, how to use results
xLAM Function Calling 60k	智能体与函数调用	60k Rows	CC BY 4.0	High-precision tool calling — every sample was verified by actually executing the API call
Hermes Function Calling v1	智能体与函数调用	12k Samples	Apache 2.0	Models that must return valid structured JSON — for agents, extraction, and tool pipelines
FineWeb 2	预训练	8TB Compressed	ODC-By 1.0	Pretraining or continued pretraining in languages other than English
Aya Dataset	多语言	204k Rows	Apache 2.0	Instruction-tuning in non-English languages with real human-written data
OpenCodeReasoning	推理	735k Samples	CC BY 4.0	Fine-tuning coding models that show their reasoning before writing the solution

如何选择数据集

让数据集匹配任务，而不是反过来。想让模型遵循指令或学会你的语气？ 使用指令/SFT 数据集——从 No Robots（人工编写）或 Dolly 15k（可商用）开始。想改进模型对答案的排序和表达？ 你需要 DPO 偏好数据集：UltraFeedback 是默认选择，HelpSteer2 提供细粒度质量评分。

要训练数学或编程「思考」模型？ 在推理轨迹上微调：OpenThoughts3 覆盖面广，NuminaMath 专注数学，OpenCodeReasoning 专注代码。要构建调用 API 的智能体？ 函数调用是一项独立技能——Glaive v2 打基础，xLAM 追求精度。非英语用户？ 指令数据用 Aya，继续预训练用 FineWeb 2。

两条最省时间的规则：训练前先查许可证（CC BY-NC 数据集不能用于商业产品）；起步规模比你想的更小——1,000 个高质量样本胜过 100,000 个嘈杂样本。拿不准时，跟着一小时微调你的第一个 LLM 的完整流程走，它用的正是这些数据集。

指令 / SFT (18)

指令数据集（SFT 数据集）是「提示–回复」对，用于监督微调——教会基础模型遵循指令、按你的格式回答或采用特定语气的第一步，也是最重要的一步。规模从 1k 条人工精选样本（LIMA）到数百万条合成混合数据（Infinity-Instruct）不等。

Python-Edu — Continued pretraining for Python code understanding (400 GB · MIT)
OpenHermes 2.5 — The default general-purpose SFT mix for 7B-13B fine-tunes (1M Rows · Apache 2.0)
Databricks Dolly 15k — Commercially safe instruction tuning - fully human-written (15k Rows · CC-BY-SA 3.0)
Stanford Alpaca — Learning the classic instruction format; research only (CC BY-NC) (52k Rows · CC-BY-NC 4.0)
ShareGPT 52K — Teaching natural multi-turn dialogue (non-commercial) (52k Conversations · CC-BY-NC 4.0)
WizardLM Evol Instruct 70k — Boosting complex-instruction handling with Evol-Instruct data (70k Rows · Apache 2.0)
LIMA: Less Is More for Alignment — Style and format alignment with a tiny curated set - quality over quantity (1k Rows · CC-BY-NC-SA 4.0)
Smoltalk — General SFT for small models (the SmolLM2 recipe) (1M Rows · Apache 2.0)
Open-Platypus — A quick STEM and logic boost on a small budget (non-commercial) (25K Questions · CC BY NC 4.0)
OpenOrca — Explanation-style SFT at scale (the Orca recipe) (4.2M Samples · MIT)
Infinity-Instruct — Large-scale general SFT when you need millions of samples (7.5M Instructions · Apache 2.0)
Magpie-Align — Fresh synthetic SFT data without seed data or scraping (3M Pairs · Apache 2.0)
Tulu 3 SFT Mix — Reproducing a state-of-the-art fully open post-training recipe (939K Samples · ODC-BY)
WebInstruct — Web-scale instruction mining for domain skills (220M Pairs · CC BY-SA 4.0)
LMSYS-Chat-1M — Studying real-world usage patterns across 25 models (non-commercial) (1M Conversations · CC BY-NC 4.0)
WildChat-1M — Training on real user conversations; safety and robustness research (1M Chats · AI2 ImpACT)
No Robots — Your first fine-tune — small, clean, 100% human-written SFT data (non-commercial license) (10k Rows · CC BY-NC 4.0)
SmolTalk 2 — Reproducing a complete modern post-training pipeline (mid-training → SFT → preference) for small models (3 Subsets (Mid 4.8M Rows) · Apache 2.0)

推理 (6)

推理数据集保存完整的思维链——得出答案的逐步「思考」过程，而不仅是最终回复。在从强推理模型（DeepSeek R1、QwQ）蒸馏出的轨迹上微调，是小型本地模型学会在回答前推演数学、代码和科学问题的方法。

MetaMathQA — Boosting GSM8K/MATH-style math skills in 7B models (395K Pairs · MIT)
Orca Math Word Problems — Grade-school math word problems for small models (200K Problems · MIT)
NuminaMath — Chain-of-thought math fine-tuning up to olympiad level (860K Problems · Apache 2.0)
OpenThoughts3-1.2M — Distilling strong math/code/science reasoning into 7B–32B models (1.2M Rows · Apache 2.0)
s1K-1.1 — Cheap, fast reasoning fine-tunes — 1k samples means minutes of training, not days (1k Rows · Apache 2.0)
OpenCodeReasoning — Fine-tuning coding models that show their reasoning before writing the solution (735k Samples · CC BY 4.0)

智能体与函数调用 (3)

智能体数据集教模型调用函数和工具：读取 JSON schema、判断何时需要调用、输出有效参数并使用返回结果。这是本地 AI 智能体的核心技能——必须专门训练，通用对话数据无法产生可靠的工具调用。

Glaive Function Calling v2 — Teaching a model basic function-calling: when to call, what JSON to emit, how to use results (113k Rows · Apache 2.0)
xLAM Function Calling 60k — High-precision tool calling — every sample was verified by actually executing the API call (60k Rows · CC BY 4.0)
Hermes Function Calling v1 — Models that must return valid structured JSON — for agents, extraction, and tool pipelines (12k Samples · Apache 2.0)

偏好（RLHF / DPO） (5)

偏好数据集包含一个提示及经过排序或评分的回复——被选中的答案和被拒绝的答案。它们驱动 RLHF、奖励模型和 DPO：在 SFT 之后教模型分辨两个看似合理的答案中人类真正偏好哪一个。应在指令微调之后使用，而不是替代它。

UltraFeedback — The default DPO preference set to run after any SFT pass (64k Rows · MIT)
Anthropic HH-RLHF — Safety-focused preference training (helpfulness and harmlessness) (170k Pairs · MIT)
DPO Mix 7K — A small, balanced DPO starter set (7k Pairs · Apache 2.0)
HelpSteer2 — Training reward models with fine-grained quality ratings (21K Samples · CC BY 4.0)
Nectar — Reward-model training with 7-way ranked responses (183K Prompts · Apache 2.0)

预训练 (4)

预训练语料是海量原始文本集合——数万亿 token 的过滤网页文本、教科书和代码——用于从零训练基础模型，或在新语言、新领域上继续预训练。消费级 GPU 无法在这些数据上微调，但它们决定了开源基础模型的知识边界。

FineWeb — Pretraining or continued pretraining on high-quality English web text (15 TB · ODC-By 1.0)
Cosmopedia — Synthetic-textbook pretraining for small models (the SmolLM recipe) (25B Tokens · Apache 2.0)
SlimPajama — Efficient English pretraining on a heavily deduplicated corpus (627B Tokens · Apache 2.0)
FineWeb 2 — Pretraining or continued pretraining in languages other than English (8TB Compressed · ODC-By 1.0)

多语言 (1)

多语言数据集把指令遵循和预训练质量带到英语之外——从 65 种语言的人工标注指令对（Aya）到 1,000 多种语言的质量过滤网页文本（FineWeb 2）。如果你的用户不用英语提问，这些必不可少。

Aya Dataset — Instruction-tuning in non-English languages with real human-written data (204k Rows · Apache 2.0)

代码 (3)

代码数据集涵盖从海量宽松许可源码语料（The Stack v2）到基于真实开源代码合成的指令对。用它们让模型专精你的技术栈、提升补全质量，或训练本地编程助手。

The Stack v2 — Pretraining code models across 600+ programming languages (900B Tokens · Various (per-file))
Magicoder-OSS-Instruct-75K — Code instruction tuning grounded in real open-source snippets (75k Rows · MIT)
CodeFeedback-Filtered-Instruction — Training coding models that respond to execution feedback (74k Rows · Apache 2.0)

视觉 (1)

视觉（多模态）数据集将图像与文本指令配对，让语言模型学会「看」。LLaVA 风格的指令集是为开源 LLM 添加图像理解能力的标准配方。

LLaVA Instruct — Adding image understanding to an open LLM (the LLaVA recipe) (158k Samples · CC-BY-NC 4.0)

评估与基准测试 (6)

评估数据集是基准测试——MMLU、HumanEval、SWE-bench——用于衡量模型，而不是训练模型。切勿混入训练数据：基准污染会虚增分数，也是严谨评审首先检查的问题。

MMLU (Massive Multitask Language Understanding) — Benchmarking general knowledge - never train on it (16k Questions · MIT)
HumanEval — Benchmarking Python code generation - never train on it (164 Problems · MIT)
Winogrande — Benchmarking commonsense reasoning (44k Problems · Apache 2.0)
MT-Bench — Judging multi-turn chat quality with LLM-as-judge (80 Questions · Apache 2.0)
SWE-bench — Benchmarking real-world software engineering - never train on it (2,294 Tasks · MIT)
MATH (Hendrycks) — Benchmarking competition math ability (12,500 Problems · MIT)

常见问题

微调聊天机器人应该用什么数据集？

从指令/SFT 数据集开始。No Robots（1 万条人工编写样本）是最干净的入门数据集，Databricks Dolly 15k 是可商用的替代选择。要改善多轮对话行为可加入 UltraChat 或 WildChat，然后用 UltraFeedback 等偏好数据集配合 DPO 打磨语气。

SFT 数据集和 DPO 数据集有什么区别？

SFT 数据集包含模型直接模仿学习的「提示–回复」对（监督微调）。DPO/偏好数据集包含一个提示加一个被选中和一个被拒绝的答案，模型学习偏好更好的那个。典型流程先 SFT 后 DPO——例如先用 Tulu 3 SFT，再用 UltraFeedback。

Alpaca 可以商用吗？

不可以——Stanford Alpaca 采用 CC BY-NC 4.0 非商业许可证，因为它由 OpenAI 的 text-davinci-003 生成。商业微调请改用 Databricks Dolly 15k（CC BY-SA 3.0）、Tulu 3 SFT Mix（ODC-BY）或 OpenHermes 2.5（MIT）。

微调数据集需要多大？

质量胜过数量。LIMA 证明 1,000 条精选样本就能对齐 65B 模型，s1K 用 1k 样本在推理任务上做到了同样的事。改变风格和格式通常 200–1,000 条优质样本就够；学习新领域技能则以 1 万–10 万条为目标。先小规模起步再评估。

什么是推理轨迹数据集？

推理轨迹数据集不仅保存最终答案，还保存完整思维链——每个解答之前的逐步思考。在轨迹上微调（OpenThoughts3、NuminaMath-CoT、OpenCodeReasoning）能把大模型的推理能力蒸馏进小型本地模型，让它们回答前先「思考」。

哪些数据集能配合 Unsloth 使用？

这里任何标准格式的数据集经过简单格式化都能用于 Unsloth：Alpaca 风格（instruction/output 列）、ShareGPT 风格（conversations）或 ChatML messages。常见的 Unsloth 入门数据集是 No Robots、Alpaca 和 Dolly 15k——我们的一小时微调指南演示了如何在 6GB GPU 上用 SFTTrainer 加载它们。