Tutorial 2: Installing Ollama

Ollama is the easiest tool to get started with local LLMs. It packages the model weights, configuration, and a runtime engine into a single binary.

1. Download & Install

Windows

1. Download Ollama for Windows.
2. Run the .exe installer.
3. Once installed, Ollama runs in the background (check your system tray).

macOS

1. Download Ollama for macOS.
2. Unzip the file and move Ollama to your Applications folder.
3. Open the application. You might need to approve it in 'Security & Privacy' settings if prompted.

Linux

Run the following command in your terminal:

2. Running Your First Model

Ollama works from the command line (Terminal on Mac/Linux, PowerShell/CMD on Windows).

To download and run the popular Llama 3.1 8B model, simply type:

Ollama will:

1. Pull the model manifest.
2. Download the model layers (approx 4.7 GB).
3. Verify the hash.
4. Drop you into a chat prompt >>>.

You can now chat! Type 'Why is the sky blue?' and hit Enter.

3. Useful Commands

4. New in Ollama v0.32: Agent Mode, Tool Calling & MLX

Ollama v0.32 is a major update for agentic workflows and Apple Silicon. The installer upgrades in place, so re-run it and confirm the version:

Built-in agent mode

v0.32 adds an interactive agent mode: a tool-calling model can now plan, call tools, and chain steps in a loop directly from the CLI — no external agent framework required. Run a model that supports tool calling and hand it a task:

With agent mode the model can invoke registered tools (shell commands, file read/write, web fetch) and keep iterating until the task is done. Everything runs on your machine, so source code and documents never leave it — the whole reason to run an agent locally instead of in the cloud. To wire up your own tools, see Building Autonomous Agents with Local LLMs.

More reliable tool calling

Tool (function) calling is far more consistent in v0.32, and Ollama speaks the OpenAI-compatible tools format on its /v1/chat/completions endpoint — so existing OpenAI-SDK code works simply by pointing the base URL at http://localhost:11434/v1. Models with strong tool support today include Qwen3-Coder, Devstral Small 2, GLM-5, Kimi K2.x, and Llama 4. Structured JSON output (JSON-schema format) is steadier too — details in Reliable JSON From Local LLMs and Setting Up a Local API Server.

MLX acceleration on Apple Silicon

On M-series Macs, v0.32 can run supported models through an MLX backend (Apple's array framework) alongside the default llama.cpp engine. MLX uses the unified-memory architecture more directly and gives noticeably faster prompt processing and token generation on M3/M4 hardware. It engages automatically for supported models on Apple Silicon and falls back to the standard engine when a model isn't MLX-ready yet — nothing to configure. On Mac, keep an eye on memory: large models still need the unified RAM to hold the weights.

5. Troubleshooting

• Error: could not connect to ollama app: Ensure the Ollama app is running in your system tray (Windows/Mac) or the service is active (Linux: sudo systemctl start ollama).
• Slow performance: Check if your model fits in your VRAM. If it's spilling into system RAM, it will be very slow.
• A 400, 404 or 413 from /v1/systemone (decision models): See Ollama /v1/systemone errors for what each status means and the fix.

6. Next Steps

Now that you have Ollama running, check out the Model Library to find models to download. Ollama 0.35 also added a /v1/systemone endpoint for decision models, which answer typed questions with a probability per option instead of text; pull Nimble or Tev1 to try one, or build a request first.