Custom Providers

A Custom Provider lets you route AI Gateway traffic to a model server you run yourself, such as Ollama, vLLM, LM Studio, or anything else that speaks the OpenAI API format. Instead of exposing that server on the public internet, it's reached through a private SocketXP tunnel.

Run the SocketXP agent on your model server

Before you can register a Custom Provider, you need a SocketXP tunnel pointed at your local model server. Install the SocketXP agent on the same machine where Ollama, LM Studio, or your other model server is running, then log in with your auth token from the SocketXP Web Portal:

sudo socketxp login <your-authtoken-goes-here>

Then create a tunnel to your model server's local port, using the --internal-url flag to make it private from the start rather than public. Ollama listens on port 11434 by default:

socketxp connect http://127.0.0.1:11434 --internal-url https://local-ollama.internal

For LM Studio, point it at whichever port its local server is running on, commonly 1234:

socketxp connect http://127.0.0.1:1234 --internal-url https://local-lmstudio.internal

The --internal-url value is a name you choose — it doesn't need to resolve anywhere on the real internet. It's just how you and the AI Gateway will refer to this tunnel afterwards, when you set the Internal Endpoint field in the next step.

Unlike the regular socketxp connect command, a HTTPS tunnel created this way is never given a public web URL of the form https://<random-id>.socketxp.com — see Why this keeps the tunnel private-only below.

For a persistent setup (running as a systemd service), the same flag is available as the internal_url field in config.json.

{
    "region": "",
    "tunnels": [
    {
        "destination": "http://127.0.0.1:11434",
        "internal_url": "https://local-ollama.internal"
    }]
}

Register it as a Custom Provider

With your tunnel running, create a Custom Provider under AI Providers in the SocketXP Web Portal. You'll need:

Field	What to set it to
`Name`	A display name for the provider.
`Slug`	Generated for you as you type the name; this becomes the routing token in `/v1/provider/<slug>/...`, and it's unique per user.
`Internal Endpoint`	The `internal_url` value you set when creating the tunnel in the previous step, e.g. `https://local-ollama.internal`.
`Local AI Model`	The local application or service the tunnel connects to.
`Provider Api Key`	Optional. Set this if your local server itself requires an API key to accept requests.
`Model Ids`	The model names your server exposes, exactly as it expects them, since custom providers are matched by an exact string, not fuzzily.

Once created, Name, Slug, Internal Endpoint, and Local AI Model are locked. Only Model Ids and Provider Api Key can be changed afterwards, and that's enforced on the server side, not just disabled in the interface.

Why this keeps the tunnel private-only

A tunnel created with internal_url never gets a public <random-id>.socketxp.com subdomain in the first place — there's no public web address to lock down after the fact, because one was never assigned. The only way to reach it is through the AI Gateway itself, by callers who present a valid AI Gateway Access Key. Your model server is never exposed on the open internet, and you get there without setting up a VPN, without a public IP, and without publishing any web URL for the server at all.

Calling a custom provider

Point your client's base_url at the gateway as usual, and set model to one of the ModelIds you registered for that provider:

from openai import OpenAI

client = OpenAI(
    base_url="https://ai-gateway.socketxp.com/v1",
    api_key="<your-access-key>",
)

response = client.chat.completions.create(
    model="llama3",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)

If you only run one server exposing a model called llama3, that's all you need.

Note:

Where this gets trickier is if you have two different custom providers that both happen to expose a model with the same name. In that case, the gateway can't tell which one you mean from model alone, so you should address the provider explicitly by its slug:

client = OpenAI(
    base_url="https://ai-gateway.socketxp.com/v1/provider/my-vllm-server",
    api_key="<your-access-key>",
)

The gateway strips the /provider/<slug> segment before forwarding, and rewrites the remaining path from /v1/provider/<slug>/chat/completions to /v1/chat/completions, matching the OpenAI-compatible mount your server is expected to expose at /v1/....

See Concepts for how Custom Providers relate to Access Keys and Policies.