Anthropic HH-RLHF — LLM Preference (RLHF / DPO) Dataset
170k human preference comparisons between two AI assistant responses, with one labeled 'chosen' (helpful, harmless) and one 'rejected'. The dataset that defined RLHF alignment for chat assistants. Used to train early Claude models and became the standard reference for alignment research and DPO fine-tuning.
Dataset Details
| Provider | Anthropic |
| Category | Preference (RLHF / DPO) |
| Size | 170k Pairs |
| License | MIT |
| Downloads | 2.8M |
| Tags | RLHF, Alignment, Safety, Helpfulness, Human-Feedback |
from datasets import load_dataset
ds = load_dataset("Anthropic/hh-rlhf")
Fine-tune with this dataset
Estimated VRAM to fine-tune with QLoRA (4-bit base model + LoRA adapters), using conservative defaults:
| 7B QLoRA | ~6GB VRAM |
| 13B QLoRA | ~10GB VRAM |
Check if your GPU can fine-tune this →
New to fine-tuning? Follow the step-by-step walkthrough: Fine-Tune Your First LLM in 1 Hour
Related datasets
- UltraFeedback — The default DPO preference set to run after any SFT pass
- HelpSteer2 — Training reward models with fine-grained quality ratings
- DPO Mix 7K — A small, balanced DPO starter set
- Nectar — Reward-model training with 7-way ranked responses
Frequently asked questions
Can I use Anthropic HH-RLHF commercially?
Yes — Anthropic HH-RLHF is released under MIT, a permissive license that allows commercial use, including training models you ship in a product. Check the dataset card for attribution requirements before release.
How much data does Anthropic HH-RLHF contain, and do I need all of it?
Anthropic HH-RLHF contains 170k Pairs. You rarely need all of it: for style and format fine-tuning, a few hundred to a few thousand examples are enough — load a slice (e.g. split="train[:1000]") and scale up only if quality plateaus.
What is Anthropic HH-RLHF best used for?
Safety-focused preference training (helpfulness and harmlessness). It belongs to the Preference (RLHF / DPO) section of our dataset hub, where you'll find alternatives and complementary sets.
← All datasets | Fine-Tuning Guide