AI gateway: the enterprise edge for models and agents

Why gateways beat raw provider keys

Applications should never hold long-lived provider secrets for every microservice. A gateway centralizes identity, policy, and observability.

Without a gateway, every team invents retries, auth, and logging differently. With a gateway, platform teams set standards once. Kimss multiplexes chat and agentic flows through one authenticated edge mapped to Azure AI Foundry projects per workspace.

See /azure-ai-foundry-agent-gateway for Foundry-specific positioning and /enterprise-ai-control-plane for the broader control-plane pillar.

Kimss gateway responsibilities

AuthN/Z, credit enforcement, workspace routing, and attribution are first-class on the Kimss edge.

• Validate X-Kimss-Key or bearer tokens.
• Enforce RBAC and workspace membership.
• Deduct or check governed requests before/as traffic flows.
• Route to the configured Foundry project or BYO endpoint.
• Record usage against agents and keys.

For internal MCP tools the gateway is dual-layer: application key, then SSO grants. See Zero-Trust AI architecture.

Optional APIM layer

Azure API Management can front AI traffic for additional policies and diagnostic logs—when enabled and verified end-to-end.

Prefer Log Analytics gateway diagnostics for immutable audit trails over deprecated synchronous custom telemetry callbacks. Keep KIMSS_AI_USE_APIM_GATEWAY disabled in production until E2E checks pass in your environment.

Architecture details live in /docs/architecture and internal compliance notes for operators.

Design checklist

Treat the gateway as production infrastructure: staging parity, key rotation, and clear failure modes when credits exhaust.

Document required headers, approved models, and escalation paths. Publish /docs/api_docs as the integrator contract—not production Swagger, which is disabled.

Frequently asked questions