API endpoints¶
The inference API accepts the requests to the models. The inference API authenticates with a token from the Tokens section of the My Account view. The {tenant} placeholder stands for the tenant, {gateway} for the gateway.
The following options are available:
| Endpoint | Description |
|---|---|
POST /v1/{tenant}/{gateway}/compat/chat/completions |
Sends a conversation to the model. The path is compatible with the schema of OpenAI. |
POST /v1/{tenant}/{gateway}/compat/completions |
Sends a single input to the model. |
POST /v1/{tenant}/{gateway}/compat/embeddings |
Creates embeddings for an input. |
POST /v1/{tenant}/{gateway}/{provider}/chat/completions |
Sends a conversation to a specified provider. |
POST /v1/{tenant}/{gateway}/{provider}/embeddings |
Creates embeddings at a specified provider. |
The admin API serves the interface. The admin API answers the following endpoints for the Tenant Administrator role.
The following options are available:
| Endpoint | Description |
|---|---|
GET/POST /admin/v1/tenants/{id}/gatewaysGET/PATCH/DELETE /admin/v1/gateways/{id} |
Manages the gateways of the tenant. Calls that write need Manage gateways and routing. |
GET/POST /admin/v1/gateways/{id}/keysDELETE /admin/v1/gateways/{id}/keys/{provider}/{alias}GET/POST/PATCH/DELETE /admin/v1/gateways/{id}/rulesDELETE /admin/v1/gateways/{id}/budget |
Manages the provider keys, the rules, and the budget of a gateway. Needs Manage gateways and routing. |
GET/POST /admin/v1/gateways/{id}/tokensPATCH/DELETE /admin/v1/gateways/{id}/tokens/{token} |
Manages the API tokens of a gateway. Needs Manage gateway API tokens. |
GET/POST /admin/v1/tenants/{id}/usersGET/PATCH/DELETE /admin/v1/users/{id}GET/POST /admin/v1/users/{id}/tokensDELETE /admin/v1/users/{id}/budgetGET /admin/v1/users/{id}/gatewaysPOST/DELETE /admin/v1/users/{id}/gateways/{gateway} |
Manages the user accounts of the tenant, their tokens, budgets, and gateway assignments. Needs Manage tenant users. |
PATCH /admin/v1/tenants/{id} |
Manages the settings of its own tenant. Needs Manage tenant settings. |
GET /admin/v1/statsGET /admin/v1/stats/timeseries |
Shows the usage figures. The Administrator and Tenant Administrator roles see the figures of the whole tenant. Every other role sees only its own. |
GET /admin/v1/stats/analytics |
Shows the analyses. Needs View analytics dashboards. |
GET /admin/v1/gateways/{id}/spendGET /admin/v1/tenants/{id}/spend |
Shows the cost of a gateway or of the tenant. Needs View cost breakdowns. |
GET /admin/v1/tenants/{id}/analytics |
Shows the analysis of the tenant. Needs Manage tenant settings. |
GET /admin/v1/logsGET /admin/v1/logs/{id}GET /admin/v1/gateways/{id}/tracesGET /admin/v1/traces/{id} |
Shows the request logs and the related traces. Needs View request logs and traces. |
GET /admin/v1/model-prices |
Lists the model prices. Needs View model prices. Writing stays reserved for the Administrator role. |
GET /admin/v1/providersGET /admin/v1/models |
Lists the providers and models supported. |
GET /admin/v1/gateways/{id}/guardrail-statsGET /admin/v1/gateways/{id}/guardrail-eventsGET /admin/v1/gateways/{id}/circuit-breaker |
Shows the hits of the content rules and the state of the circuit breaker. |
GET /healthz |
Reports the operational state of the gateway. |
Note
For this role, the API answers the POST /admin/v1/tenants, DELETE /admin/v1/tenants/{id}, and GET /admin/v1/audit-log endpoints and the endpoints under /admin/v1/model-prices that write with the 403 status.
Note
For the AI Manager role, the admin API answers only GET /admin/v1/stats, GET /admin/v1/stats/timeseries, GET /admin/v1/stats/analytics, GET /admin/v1/providers, and GET /admin/v1/models. The admin API answers every other endpoint with the 403 status. For the Finance role, it answers only GET /admin/v1/stats and GET /admin/v1/stats/timeseries. This role creates no tokens, and the inference API is therefore not available to it.
The subsections describe each endpoint separately with its parameters, an example, and its status codes.