API Access: Ollama Speaks OpenAI

The nice thing about Ollama: it offers an OpenAI-compatible API. So if you already have code that talks to the OpenAI API, you only need to change the base URL and swap the model name.

curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3.3:8b",
    "messages": [
      {"role": "user", "content": "Reply with OK"}
    ]
  }'

That's exactly the same structure as OpenAI -- just without the Authorization header (local, no key needed) and with a different model name. Tools like n8n, LangChain, and LlamaIndex support Ollama as a drop-in.

Which Model for Which Purpose

Here's an honest assessment from our own testing, as of April 2026:

Test two models side by side against your real tasks. Benchmarks tell you little. Only your use cases decide.

Performance Reality Check

The uncomfortable truth: local models are slower than cloud models. Not in model quality, but in throughput. While GPT-4o on an Azure server delivers roughly 100 tokens per second, Llama 3.3 70B on an RTX 4090 runs at about 15 to 25 tokens per second.

What that means in practice: a response that takes two seconds in the cloud takes eight to twelve seconds locally. For asynchronous agents (email triage, document classification) that's irrelevant. For real-time chat with customers, it's noticeable.

If speed is critical, you either need better hardware (multiple GPUs in parallel, inference-optimized cards like A100/H100) or you combine local and cloud: local for sensitive data, cloud for non-critical fast responses.

What the GDPR Documentation Says (and Why It's Shorter)

The big advantage of Ollama for your DPIA: the data processor section disappears. There is no external processor. You process the data yourself, on your own premises, on your own infrastructure.

What you have to document more than with cloud: the technical and organizational measures (TOMs) for your server. Who has physical access? How is patching handled? What are the backups like? That's standard IT security, applies to any server, isn't AI-specific -- but it has to be in the DPIA.

When You Should NOT Go Local

Three situations where we at kiba advise against local:

When nobody on your team knows Linux or server operations. Ollama isn't "click and it runs." It runs at first. Six months later, when a driver update breaks something or the server has to reboot, you need someone who knows what they're doing. If that's nobody and you have no external IT partner, cloud is the more practical path despite the weaker GDPR position.

When your volume is below 100,000 tokens per day. At 2026 token prices, that's less than EUR 50 per month in the cloud. A EUR 4,000 hardware investment never pays off there. Cloud is cheaper when volume is small.

When you need maximum model quality. Local models in 2026 are good, but GPT-4o, Claude Opus 4, and Gemini 2.5 Pro are still a step ahead. If your use case demands the very best reasoning -- complex contract analysis, multi-step planning tasks -- local works, but you're giving up quality.

Your First Local Agent

Once the setup is in place, building your first agent is no different than in the cloud. You use n8n, LangChain, or your own script, point it at http://localhost:11434/v1 instead of OpenAI, and the workflow is identical.

We'll lay out the concrete blueprint for a first agent -- with trigger, steps, outputs -- in the next article in the series. There we use n8n as the orchestrator and can plug in either local or cloud as the backend.