Your First Local AI: Get Up and Running in 15 Minutes

You've read about local AI. You understand what it is and why people use it. Now you want to actually try it.

This guide gets you from zero to a working local AI conversation in about 15 minutes, with no technical background required. We'll use a free tool called Ollama — the most beginner-friendly way to run local AI on Mac, Windows, or Linux.

What You'll Need

• A computer with at least 8 GB of RAM (16 GB recommended for a better experience)
• About 5 GB of free disk space for the model file
• An internet connection for the initial download (after that, everything runs offline)

That's it. No special hardware. No accounts to create. No credit card.

Step 1: Install Ollama (5 minutes)

Ollama is free software that handles all the complexity of running local AI — downloading models, managing them, and running them efficiently on your hardware.

On Mac: Go to ollama.com and click the Download button. Open the downloaded .dmg file, drag Ollama into your Applications folder, and open it. You'll see the Ollama icon appear in your menu bar. On Windows: Go to ollama.com and click Download for Windows. Run the installer (.exe file). Ollama installs and starts automatically. You'll see it in your system tray. On Linux: Open a terminal and run:

curl -fsSL https://ollama.com/install.sh | sh

Once installed, Ollama runs quietly in the background. You don't need to open any app — it's ready to use from the terminal (command line).

Step 2: Open Your Terminal

The easiest way to use Ollama for the first time is through the terminal — a text window where you type commands. Don't worry, the commands are simple.

On Mac: Press Command + Space, type "Terminal", press Enter. A black or white window with a text prompt opens. On Windows: Press Windows key, type "Command Prompt" or "PowerShell", press Enter. On Linux: You probably already know where your terminal is.

You'll see a blinking cursor waiting for you to type. This is where you'll run Ollama commands.

Step 3: Download and Run Your First Model (5–10 minutes)

Type this command and press Enter:

ollama run llama3.2

The first time you run this, Ollama downloads the Llama 3.2 model — a capable 3 billion parameter model made by Meta that's about 2 GB in size. This takes 2–5 minutes depending on your internet speed. You'll see a progress bar.

Once it's downloaded, it switches directly into chat mode. You'll see:

>>> Send a message (/? for help)

You're in. Type anything and press Enter.

Why Llama 3.2 for Your First Model?

Llama 3.2 3B is small enough to run on almost any computer (needs about 3 GB RAM while active), downloads quickly, and is genuinely capable for writing, Q&A, and basic coding tasks. It's a good first experience because it just works without needing to think about hardware compatibility.

After you've tried it, you can move on to more capable models.

Step 4: Try It Out

Here are some things to try in your first session:

Simple Q&A:

>>> What are three interesting things about the history of the internet?

Writing help:

>>> Rewrite this sentence to sound more professional: "hey just wanted to check in about that thing we talked about"

Explain something:

>>> Explain what inflation is like I'm 12 years old

Brainstorm:

>>> Give me 5 creative names for a bakery that specializes in sourdough bread

Code help:

>>> Write a simple Python script that reads a text file and counts how many times each word appears

When you're done, type /bye and press Enter, or just close the terminal window.

Step 5: Try a More Capable Model (Optional)

Llama 3.2 3B is a good start, but if your computer has 16 GB of RAM, you can run a significantly more capable model with one command.

ollama run llama3.2:8b

This downloads the 8 billion parameter version — about 5 GB. It's noticeably smarter on complex tasks, handles longer and more nuanced responses, and is the sweet spot for most everyday AI use on a standard laptop.

If you have a modern Mac with Apple Silicon (M1, M2, M3, or M4), try:

ollama run qwen3.5:14b

This is a 14 billion parameter model from Alibaba's research lab — one of the best local models available in 2026, and Apple Silicon runs it remarkably quickly thanks to its unified memory architecture.

Managing Your Models

A few useful Ollama commands to know:

# See all models you've downloaded
ollama list

# Download a model without starting a chat
ollama pull mistral

# Remove a model to free disk space
ollama rm llama3.2

# See what's currently running
ollama ps

Models are stored on your computer and persist between sessions. Once downloaded, they run instantly — no waiting for another download.

Getting a Better Chat Interface (Optional)

The terminal chat is functional, but many people prefer a browser-based interface that looks more like ChatGPT. The most popular option is Open WebUI — a free, self-hosted interface that runs on your computer and connects to Ollama.

If you have Docker installed, Open WebUI installs with one command. But even without it, Ollama's built-in terminal interface is perfectly usable for getting started. You can always add a nicer interface later.

What to Expect vs. Cloud AI

Here's what's genuinely different compared to ChatGPT or Claude:

Slower responses: Local AI generates text at 5–50 words per second depending on your hardware and model size. Cloud AI, running on specialized hardware, often feels near-instant. The quality of the output is similar; the speed isn't. This is normal. No memory between sessions: By default, each new ollama run session starts fresh with no memory of past conversations. (This can be changed with more advanced setups, but out of the box, sessions are independent.) No internet access: The model knows only what it learned during training — it doesn't search the web. For current events, today's news, or recent information, you'd need to paste relevant information into the conversation yourself. Completely private: Nothing you type goes anywhere. The model runs entirely on your machine. Completely free to use as much as you want: No subscription, no usage limits, no "you've hit your message cap." Run it for 10 hours straight if you want.

Choosing Your Next Models

Once you've spent time with Llama 3.2, you might want to explore:

For better general quality (if you have 16 GB RAM):

ollama run llama3.2:8b     # Meta's 8B model, very capable
ollama run gemma3:9b       # Google's 9B model, excellent at reasoning

For coding specifically:

ollama run qwen2.5-coder:7b   # Specialized for code, excellent
ollama run deepseek-coder:6.7b # Strong on debugging and code generation

For maximum quality (if you have 32 GB+ RAM):

ollama run qwen3.5:14b        # Excellent all-around model
ollama run mistral-small3.1   # Very capable, good for long documents

Each model has different strengths. Part of the fun of local AI is trying different ones for different tasks and discovering which you like best for your work.

You're Done

That's the entire setup. You now have a working local AI on your computer:

• Completely private — nothing leaves your machine
• Completely free — no subscriptions, no usage limits
• Works offline — no internet needed after the initial download
• Yours to customize and experiment with

The terminal interface is the fastest way to get started, but it's just the beginning. There are browser-based interfaces, integrations with text editors and IDEs, ways to connect local AI to your own documents, and much more — all built on the same Ollama foundation you just installed.

For now, just talk to it. Ask it things. See what it can and can't do. The best way to understand local AI is to use it.

---

Find your next model to try → Model Library Check what larger models your hardware can run → Hardware Analyzer