Setting up a Local API Server

Written by Jakub Rusinowski · Last updated July 10, 2026

Turn your local machine into an AI backend for other apps.

In This Guide

• Ollama API
• OpenAI Compatible Endpoint
• Going Further

Turn your local machine into an AI backend for other apps.

Ollama API

Ollama runs a server by default on port 11434.

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "Why is the sky blue?"
}'

OpenAI Compatible Endpoint

Many apps expect the OpenAI API format. Ollama provides this too! Endpoint: http://localhost:11434/v1/chat/completions

Python Example

from openai import OpenAI

client = OpenAI(
    base_url='http://localhost:11434/v1',
    api_key='ollama', # required but unused
)

response = client.chat.completions.create(
    model="llama3.1",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Going Further

• → Personal AI Home Server — run this API always-on and reach it from any device via Tailscale.
• → Reliable JSON & Structured Output — make the API safe to build apps on.
• → vLLM Setup Guide — when one user becomes many, serve at production throughput.
• → Can't Connect? Connection Refused & Port Fixes

← All Guides | Check GPU Compatibility