Your private model

Chapter 2 of A private model for your agent. Draft: tell us how it goes.

The model runs in Ollama, a popular open-source program for running AI models on your own computer. Edgible's Ollama recipe for your system installs it to use the computer's GPU and puts a key in front of it.

Two details are worth knowing, because they're why this is safe:

• Ollama listens on that computer only. Nothing else on your network, not another computer, a guest on your wifi, or anything that gets into one, can reach it directly.
• The address Edgible gives it is a small guard in front. It lets a caller use the model and nothing else: no downloading, deleting or changing models. A key can use your models and can't change them. You change them on that computer.

What you need

• Chapter 1 done, and Claude Code on that computer.
• Free memory for the model you pick, and a few gigabytes of disk. The table on the front page says what suits your computer. If you're unsure, start with qwen2.5:7b.

1. Ask Claude Code to set it up

In Claude Code on the GPU computer, say:

Set up Ollama on this computer so my AI gateway can use it privately.

It tells you what it's about to install, and asks first: Ollama and Caddy (the small guard), each checked against its publisher's checksum before anything is unpacked, and two background services that start when you log in. It then publishes the guard on Edgible as an app, ollama, that needs a key.

If another Ollama is already running on this computer, it says so and picks a free port rather than taking over. Your existing models can be reused: say so when it asks where models live.

2. Download a model

Download the qwen2.5:7b model.

It downloads onto that computer, on your own connection, and takes a few minutes. A bigger model takes longer: gpt-oss:20b is 13 GB. Download a second, larger one now if it has the memory, for private-smart:

Download the gpt-oss:20b model too.

3. Check it works, and is locked

Check my private model runs on the GPU, and that its Edgible address needs a key.

It asks the model a tiny question on the computer and confirms it ran entirely on the GPU. Then it calls the Edgible address without a key, and expects to be turned away. It should tell you it was.

4. Make a key for your gateway

Make an Edgible API key for my ollama app, for my AI gateway to use, and keep it safe in my Edgible account.

Name the ollama app, so it can't mean something else: a key for an app that calls the gateway is a different thing, with its own budget. Claude tells you what it's about to do, something like "make a key named ai-gateway for the ollama app and store it as the secret ollama", and asks. You never see the key and it never goes into the chat. If the gateway is ever compromised, you delete that one key and nothing else changes.

Already have a key stored from an earlier try? Claude says so, and warns that making a new one deletes the old: anything still using it stops until it's given the new one. In chapter 3, adding Ollama to the gateway gives it the new key and restarts it.

What you've just done

You have a private AI model, and it's well guarded:

• It runs on your hardware. What you ask it never leaves your computer.
• Only a key opens it, and each key can be named, given an end date, and switched off alone.
• A key can use it but not change it. Nobody holding a key can download, delete or replace your models.
• Nothing on your network can reach it directly, including other computers at home.
• Its address is on Edgible, so your gateway, on another machine, can reach it from anywhere.