G4

Guide

Gemma 4 + Ollama Usage Guide

The complete guide to running Gemma 4 with Ollama — install, pull models, use the REST API, integrate with Python via OpenAI SDK, create custom Modelfiles, and more.

Ollama REST API OpenAI SDK Modelfile

1. Install Ollama

# Linux / macOS
curl -fsSL https://ollama.com/install.sh | sh

# macOS (Homebrew)
brew install ollama

# Windows: download .exe from ollama.com

After installation, Ollama runs as a background service. Verify with ollama --version.

2. Pull Gemma 4 Models

# Pull the default 31B model
ollama pull gemma4

# Pull specific variants
ollama pull gemma4:e4b    # Edge 4B — best for 8 GB VRAM
ollama pull gemma4:e2b    # Edge 2B — runs on 4 GB VRAM or CPU

# List downloaded models
ollama list

Which Variant to Pull

TagVRAMSpeed
gemma4:e2b~4 GBFastest
gemma4:e4b~6 GBFast
gemma4~18 GBBest quality

Ollama uses GGUF quantized models (Q4_K_M by default).

3. Run via CLI

# Interactive chat in terminal
ollama run gemma4

# Single prompt (non-interactive)
ollama run gemma4 "Explain the MoE architecture in Gemma 4"

# With a custom system prompt
ollama run gemma4 --system "You are a Python expert." "Write a FastAPI hello world"

In interactive mode, type /bye to exit or /help to see commands.

4. REST API

Ollama exposes a REST API at http://localhost:11434:

# The Ollama REST API runs at localhost:11434
# Generate completion
curl http://localhost:11434/api/generate -d '{
  "model": "gemma4",
  "prompt": "Why is Gemma 4 good for local deployment?",
  "stream": false
}'

# Chat completions
curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [
    {"role": "user", "content": "Hello!"}
  ]
}'

POST /api/generate

Single completion

POST /api/chat

Multi-turn conversation

GET /api/tags

List installed models

5. Use with OpenAI Python SDK

Ollama supports the OpenAI API format at /v1/ — use your existing OpenAI code with zero changes:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # required but unused
)

response = client.chat.completions.create(
    model="gemma4",
    messages=[{"role": "user", "content": "What is Gemma 4?"}]
)
print(response.choices[0].message.content)

6. Streaming Responses

import requests
import json

response = requests.post(
    "http://localhost:11434/api/chat",
    json={
        "model": "gemma4",
        "messages": [{"role": "user", "content": "Tell me a story"}],
        "stream": True
    },
    stream=True
)

for line in response.iter_lines():
    if line:
        chunk = json.loads(line)
        print(chunk["message"]["content"], end="", flush=True)

7. Custom Modelfile

From Ollama Base Model

# Create a Modelfile
FROM gemma4:e4b
SYSTEM "You are a helpful coding assistant specializing in Python."
PARAMETER temperature 0.7
PARAMETER top_p 0.9
# Build and run your custom model
ollama create mygemma -f Modelfile
ollama run mygemma

From Local GGUF File

# Use a local GGUF file
FROM /path/to/your/gemma4-q4_k_m.gguf
SYSTEM "You are a helpful assistant."

Download GGUF files from community repos like bartowski/google_gemma-4-E4B-it-GGUF on Hugging Face.

Tips & Common Issues

Performance Tips

  • Set OLLAMA_NUM_GPU=1 to force GPU offloading
  • Use OLLAMA_NUM_PARALLEL=4 for concurrent requests
  • Keep context short — 2048 tokens is enough for most tasks
  • E4B with Q4_K_M is the best quality/speed ratio under 8 GB

Expose Ollama on Network

OLLAMA_HOST=0.0.0.0 ollama serve

Then access from other devices at http://<your-ip>:11434. Add authentication with a reverse proxy (nginx/Caddy) for production.

Related Guides