WebInstruct — LLM 指令 / SFT Dataset

220M instruction-response pairs extracted from the web by identifying educational content using a recall-then-verify pipeline. Covers K-12 through graduate-level content across math, science, and engineering — enabling large-scale instruction fine-tuning without human curation.

Dataset Details

from datasets import load_dataset
ds = load_dataset("TIGER-Lab/WebInstructSub")

用这个数据集微调

使用 QLoRA（4-bit 基础模型 + LoRA 适配器）微调的预计显存需求（保守默认参数）：

检测你的 GPU 能否微调 →

微调新手？跟着分步教程走： 一小时微调你的第一个 LLM

相关数据集

• OpenHermes 2.5 — The default general-purpose SFT mix for 7B-13B fine-tunes
• Infinity-Instruct — Large-scale general SFT when you need millions of samples
• Python-Edu — Continued pretraining for Python code understanding
• Databricks Dolly 15k — Commercially safe instruction tuning - fully human-written

常见问题

← 全部数据集 | Fine-Tuning Guide