Turn your local machine into an AI backend for other apps.
Ollama API
Ollama runs a server by default on port 11434. Since version 0.35 the same server also answers /v1/systemone for decision models: build a request, follow the setup guide, and see what a 400, 404 or 413 means.
curl http://localhost:11434/api/generate -d '{
"model": "llama3.1",
"prompt": "Why is the sky blue?"
}'OpenAI Compatible Endpoint
Many apps expect the OpenAI API format. Ollama provides this too! Endpoint: http://localhost:11434/v1/chat/completions
Python Example
from openai import OpenAI
client = OpenAI(
base_url='http://localhost:11434/v1',
api_key='ollama', # required but unused
)
response = client.chat.completions.create(
model="llama3.1",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)Going Further
- → Personal AI Home Server — run this API always-on and reach it from any device via Tailscale.
- → Reliable JSON & Structured Output — make the API safe to build apps on.
- → vLLM Setup Guide — when one user becomes many, serve at production throughput.
- → Inference Engines Compared — Ollama, llama.cpp, LM Studio, vLLM, SGLang and more: which server for which job.
- → Can't Connect? Connection Refused & Port Fixes