Orca Math Word Problems
200K diverse grade-school math word problems synthetically generated by GPT-4 using an agent-based approach. Each problem is uniquely crafted without duplication from existing datasets, achieving state-of-the-art results with small 7B fine-tunes.
from datasets import load_dataset
ds = load_dataset("microsoft/orca-math-word-problems-200k")Preview a sample row
{
"question": "A bakery makes 250 cookies per batch and bakes 4 batches per day. They sell 80% of their cookies each day. How many cookies remain unsold?",
"answer": "Total cookies = 250 × 4 = 1000. Sold = 1000 × 0.80 = 800. Remaining = 1000 − 800 = 200 cookies."
}Fine-tune with this dataset
Estimated VRAM to fine-tune with QLoRA (4-bit base model + LoRA adapters), using conservative defaults:
New to fine-tuning? Follow the step-by-step walkthrough: Fine-Tune Your First LLM in 1 Hour
Frequently asked questions
Can I use Orca Math Word Problems commercially?
Yes — Orca Math Word Problems is released under MIT, a permissive license that allows commercial use, including training models you ship in a product. Check the dataset card for attribution requirements before release.
How much data does Orca Math Word Problems contain, and do I need all of it?
Orca Math Word Problems contains 200K Problems. You rarely need all of it: for style and format fine-tuning, a few hundred to a few thousand examples are enough — load a slice (e.g. split="train[:1000]") and scale up only if quality plateaus.
What is Orca Math Word Problems best used for?
Grade-school math word problems for small models. It belongs to the Reasoning section of our dataset hub, where you'll find alternatives and complementary sets.