Running LLMs on Laptops & Raspberry Pi

Written by Jakub Rusinowski · Last updated July 10, 2026

You don't need a $2000 GPU to run AI. Small Language Models (SLMs) are efficient enough for edge devices.

In This Guide

• Recommended Models
• Raspberry Pi 5 Setup
• Laptop Optimization (Battery Life)

You don't need a $2000 GPU to run AI. Small Language Models (SLMs) are efficient enough for edge devices.

Recommended Models

• Phi-3.5 Mini (3.8B): Microsoft's marvel. Runs on almost anything.
• Qwen 2.5 1.5B/3B: Incredible coding performance for the size.
• Gemma 2 2B: Google's lightweight model.

Raspberry Pi 5 Setup

1. Install a 64-bit OS (Ubuntu or Raspberry Pi OS). 2. Install Ollama: curl -fsSL https://ollama.com/install.sh | sh 3. Run a tiny model: ollama run qwen2.5:1.5b 4. Expectation: 3-5 tokens per second. Usable for simple commands.

Laptop Optimization (Battery Life)

Running LLMs drains battery fast.

• Use GGUF format: It is optimized for CPU efficiency.
• Limit Threads: In Ollama, you can set OMP_NUM_THREADS=4 to prevent it from using all cores, saving power.

← All Guides | Check GPU Compatibility