API endpoints

The inference API accepts the requests to the models. The inference API authenticates with a token from the Tokens section of the My Account view. The {tenant} placeholder stands for the tenant, {gateway} for the gateway.

The following options are available:

Endpoint	Description
POST /v1/{tenant}/{gateway}/compat/chat/completions	Sends a conversation to the model. The path is compatible with the schema of OpenAI.
POST /v1/{tenant}/{gateway}/compat/completions	Sends a single input to the model.
POST /v1/{tenant}/{gateway}/compat/embeddings	Creates embeddings for an input.
POST /v1/{tenant}/{gateway}/{provider}/chat/completions	Sends a conversation to a specified provider.
POST /v1/{tenant}/{gateway}/{provider}/embeddings	Creates embeddings at a specified provider.

The admin API serves the interface. The admin API answers the following endpoints for the Administrator role.

The following options are available:

Endpoint	Description
GET/POST /admin/v1/tenants GET/PATCH/DELETE /admin/v1/tenants/{id}	Manages the tenants. The creation needs Create tenants, the change and the deletion need Manage tenant settings, the permanent deletion needs Purge tenants.
GET/POST /admin/v1/tenants/{id}/gateways GET/PATCH/DELETE /admin/v1/gateways/{id}	Manages the gateways of a tenant. Calls that write need Manage gateways and routing.
GET/POST /admin/v1/gateways/{id}/keys DELETE /admin/v1/gateways/{id}/keys/{provider}/{alias} GET/POST/PATCH/DELETE /admin/v1/gateways/{id}/rules DELETE /admin/v1/gateways/{id}/budget	Manages the provider keys, the rules, and the budget of a gateway. Needs Manage gateways and routing.
GET/POST /admin/v1/gateways/{id}/tokens PATCH/DELETE /admin/v1/gateways/{id}/tokens/{token}	Manages the API tokens of a gateway. Needs Manage gateway API tokens.
GET/POST /admin/v1/tenants/{id}/users GET/PATCH/DELETE /admin/v1/users/{id} GET/POST /admin/v1/users/{id}/tokens DELETE /admin/v1/users/{id}/budget GET /admin/v1/users/{id}/gateways POST/DELETE /admin/v1/users/{id}/gateways/{gateway}	Manages the user accounts of the tenant, their tokens, budgets, and gateway assignments. Needs Manage tenant users.
GET /admin/v1/stats GET /admin/v1/stats/timeseries	Shows the usage figures. The Administrator and Tenant Administrator roles see the figures of the whole tenant. Every other role sees only its own.
GET /admin/v1/stats/analytics	Shows the analyses. Needs View analytics dashboards.
GET /admin/v1/gateways/{id}/spend GET /admin/v1/tenants/{id}/spend	Shows the cost of a gateway or of a tenant. Needs View cost breakdowns.
GET /admin/v1/tenants/{id}/analytics	Shows the analysis of a tenant. Needs Manage tenant settings.
GET /admin/v1/logs GET /admin/v1/logs/{id} GET /admin/v1/gateways/{id}/traces GET /admin/v1/traces/{id}	Shows the request logs and the related traces. Needs View request logs and traces.
POST /admin/v1/playground/token GET /admin/v1/playground/search GET /admin/v1/playground/trace/{id}	Operates the playground. The retrieval of a trace needs View request logs and traces.
GET/PUT /admin/v1/model-prices DELETE /admin/v1/model-prices/{provider}/{model}	Manages the model prices. Reading needs View model prices, writing needs Manage model prices and deprecations.
GET /admin/v1/providers GET /admin/v1/models	Lists the providers and models supported.
GET /admin/v1/gateways/{id}/guardrail-stats GET /admin/v1/gateways/{id}/guardrail-events GET /admin/v1/gateways/{id}/circuit-breaker	Shows the hits of the content rules and the state of the circuit breaker.
GET /admin/v1/audit-log	Shows the audit log of the platform. Needs View the platform audit log.
POST/GET /admin/v1/client-errors	Accepts the errors of the interface. Reading needs Triage feedback, reports, and client errors.
GET /healthz	Reports the operational state of the gateway.

The subsections describe each endpoint separately with its parameters, an example, and its status codes.