Ollama: llama.cpp Wrapped as a Server
What actually happens when you type `ollama pull`: GGUF weights, a llama.cpp runtime, and an OpenAI-shaped API on localhost:11434. Serve Gemma 4 E4B on your own machine and point any OpenAI client at it.
What you'll learn
- Understand the Ollama = llama.cpp + GGUF + HTTP server stack
- Pull and serve gemma4:e4b locally
- Reuse the OpenAI client against a local base_url
The stack, layer by layer
"Ollama" is not a model — it is four thin layers stacked on top of each other. Click each one:
HTTP server on localhost:11434
"ollama serve" wraps llama.cpp in a long-running HTTP server. Models stay loaded between requests, and the endpoints copy the OpenAI API shape — /v1/chat/completions and friends — on port 11434, on your own machine.
Pull and serve
# --- install Ollama and start it in the background --- !curl -fsSL https://ollama.com/install.sh | sh import subprocess, time, os # Use 0.0.0.0 to ensure the server is accessible within the Colab network os.environ["OLLAMA_HOST"] = "0.0.0.0:11434" subprocess.Popen(["ollama", "serve"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) time.sleep(10) #use gemma 4 model for agentic tool calling !ollama pull gemma4:e4b
One function the whole course reuses
This is the exact snippet from the notebook's setup cell — the model object every lesson's agent is built on. Note what it is: the ordinary OpenAI client with one URL swapped.
# --- one model object per size. `model` is the one the lessons use. ---
import nest_asyncio; nest_asyncio.apply() # lets `await` work inside a notebook
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
def ollama_model(name):
# Ollama copies the OpenAI API shape, so we reuse the OpenAI client and swap the URL.
return OpenAIChatModel(
name,
provider=OpenAIProvider(base_url="http://localhost:11434/v1", api_key="ollama"),
)
model = ollama_model("gemma4:e4b")
print("setup done")That URL swap is the entire "local first" strategy: develop against a free local Gemma 4 E4B, and if you ever need a bigger cloud model, change base_url back — zero other code changes. Now the brain is served; the capstone puts a production API in front of it.