Your AI gateway

Chapter 3 of Your own AI agent, from anywhere. New: tested on our own machines, so tell us how it goes.

Before you run an agent, put the guardrails in. An AI gateway sits between your agents and the AI providers, and gives you:

• One place for your keys. You store each provider's key once. Your agents never see them: each gets its own gateway key instead.
• A budget per agent. Each agent can spend only its own budget, say five dollars a month. A runaway agent stops at its limit, not at your credit card's.
• The real dollars. What each agent spent, on which model, from your phone.
• Models by purpose, not by name. Agents ask for fast, smart or best, and you decide which model each one means, and change it any time.
• A backup. When a provider has an outage or a free tier runs out, the next one answers.

The gateway is LiteLLM, the most widely used open-source AI gateway, in a version we've tested.

You'll set the gateway up first, then add the AI providers you want to use, one at a time, just by asking.

What you need

• Chapters 1 and 2 done, on a machine with 4 GB of memory or more.
• An account with at least one AI provider, and a little credit. Any of OpenAI, Anthropic, Google Gemini, DeepSeek, Mistral, Groq or OpenRouter will do. The examples use DeepSeek, which is inexpensive; swap in yours.

1. Set up the gateway

In Claude, say:

Set up an AI gateway on my edgible-vm, called ai-gateway.

It checks the machine, then asks you one thing: whether to give the gateway an address your apps can call. Say yes. That address answers only apps holding a key the gateway gave them, and each app gets its own key and budget. Your connector keeps the gateway's own master key, so you never handle it.

The gateway is running, but it can't answer yet: it has no AI provider. Next.

2. Add your first provider

Add my DeepSeek key to the gateway.

If you stored that key with Edgible before, Claude simply uses it. Otherwise it gives you a key link. Open it, paste the key from the provider's page, and save. The key goes straight to your Edgible account, never into the chat. Links work once, for ten minutes, and from your phone too.

Claude then tells you what it's about to do and asks, something like:

I'll add DeepSeek for fast and smart. The gateway restarts and doesn't answer for about a minute. OK?

Say yes. When it's back, Claude checks each purpose answers, and reports what it did, in words like these:

Added DeepSeek. fast uses deepseek-chat, smart uses deepseek-reasoner. Both answered a test question. Not set up: best, open, private. To change one, say "make smart use …".

You never had to choose a model name or a purpose. A first provider fills fast and smart, so you have a gateway that works.

3. Check it answers

Check my AI gateway answers.

It sends one tiny question through the gateway, and tells you which provider answered, how long it took, and what it cost: a tiny fraction of a cent.

4. Add another provider, any time

Say which provider, in the same way:

Add my Gemini key to the gateway.

Claude is careful not to change what already works. With DeepSeek already answering, it tells you something like:

Added Gemini as fast's backup. fast and smart already use DeepSeek, so I left them. Say "make smart use Gemini" to switch.

So adding a provider is safe: it never quietly sends your questions somewhere new. When you want it to, say so:

Make smart use Gemini.

Use OpenAI for fast and smart.

The most expensive purpose, best, is never filled unless you ask:

Add my Anthropic key to the gateway, for best.

Each change restarts the gateway for about a minute.

The purposes

Change what any of them means by asking: "Make smart use Gemini Pro." A purpose that no provider serves yet can't answer, and Claude's report tells you which those are. fast gets a backup when you add a second provider.

What you've just done

You've put the guardrails in before letting an agent loose, which is exactly the right order:

• Your keys are in one safe place, stored once, never written into a chat, a file or an app.
• You chose the providers, and can add or switch them in a sentence, without touching a model name unless you want to.
• Every agent you add gets its own key and budget. If one misbehaves, it stops at its limit, and you can switch it off without touching the others.
• You'll see the real dollars, by agent and by model, whenever you ask.
• It's yours. No middleman between your agents and the providers, no markup, and nobody else's dashboard.

Most people running agents have none of this, and find out at the end of the month.