OpenOrca — LLM 指令 / SFT Dataset

A 4.2M sample dataset replicating Microsoft Research's Orca paper by augmenting FLAN Collection with GPT-4 and GPT-3.5 explanations. Enables small models to match much larger models through explanation-based fine-tuning.

Dataset Details

from datasets import load_dataset
ds = load_dataset("Open-Orca/OpenOrca")

用这个数据集微调

使用 QLoRA（4-bit 基础模型 + LoRA 适配器）微调的预计显存需求（保守默认参数）：

检测你的 GPU 能否微调 →

微调新手？跟着分步教程走： 一小时微调你的第一个 LLM

相关数据集

• OpenHermes 2.5 — The default general-purpose SFT mix for 7B-13B fine-tunes
• WizardLM Evol Instruct 70k — Boosting complex-instruction handling with Evol-Instruct data
• Python-Edu — Continued pretraining for Python code understanding
• Databricks Dolly 15k — Commercially safe instruction tuning - fully human-written

常见问题

← 全部数据集 | Fine-Tuning Guide