Custom Providers

A Custom Provider lets you route AI Gateway traffic to a model server you run yourself, such as Ollama, vLLM, LM Studio, or anything else that speaks the OpenAI API format. Instead of exposing that server on the public internet, it's reached through a private SocketXP tunnel.

Setting one up

Create a Custom Provider under AI Providers in the SocketXP Web Portal. You'll need:

Field	What to set it to
Name	A display name for the provider.
Slug	Generated for you as you type the name; this becomes the routing token in /v1/provider/<slug>/..., and it's unique per user.
Internal Endpoint	The SocketXP tunnel your model server is running behind.
Local AI Model	The local application or service the tunnel connects to.
Provider Api Key	Optional. Set this if your local server itself requires an API key to accept requests.
Model Ids	The model names your server exposes, exactly as it expects them, since custom providers are matched by an exact string, not fuzzily.

Once created, Name, Slug, Internal Endpoint, and Local AI Model are locked. Only Model Ids and Provider Api Key can be changed afterwards, and that's enforced on the server side, not just disabled in the interface.

Why the secure tunnel becomes private-only

Setting Internal Endpoint on a tunnel changes its behavior: it's no longer reachable through its normal public web URL https://xyz.socketxp.com subdomain. The only way to reach it is through the AI Gateway itself. That's deliberate. It means the model server behind it is never exposed on the open internet, only to callers who present a valid AI Gateway Access Key, and you get there without setting up a VPN, without a public IP, and without publishing any web URL for the server at all.

Calling a custom provider

Point your client's base_url at the gateway as usual, and set model to one of the ModelIds you registered for that provider:

from openai import OpenAI

client = OpenAI(
    base_url="https://ai-gateway.socketxp.com/v1",
    api_key="<your-access-key>",
)

response = client.chat.completions.create(
    model="llama3",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)

If you only run one server exposing a model called llama3, that's all you need. Where this gets trickier is if you have two different custom providers that both happen to expose a model with the same name. In that case, the gateway can't tell which one you mean from model alone, so you should address the provider explicitly by its slug:

client = OpenAI(
    base_url="https://ai-gateway.socketxp.com/v1/provider/my-vllm-server",
    api_key="<your-access-key>",
)

The gateway strips the /provider/<slug> segment before forwarding, and rewrites the remaining path from /v1/provider/<slug>/chat/completions to /v1/chat/completions, matching the OpenAI-compatible mount your server is expected to expose at /v1/....

See Concepts for how Custom Providers relate to Access Keys and Policies.