Which LLM providers and protocols does Kong AI Gateway support?

It supports a unified API for OpenAI, Anthropic, Azure OpenAI, Bedrock, Mistral, Cohere, local models, and MCP servers, allowing multi-LLM routing and failover without changing application code.

Is Kong AI Gateway open source and self-hostable?

Yes, the OSS Gateway plus AI plugins is free to self-host on Linux, Docker, or Kubernetes. A managed control plane (Kong Konnect) and Enterprise features like SSO and RBAC are available via paid plans.

Who is the ideal user for this tool?

It is best for platform and security teams running production LLM workloads who already use Kong or need an enterprise-grade proxy for multi-team governance and agent-to-agent observability.

Kong AI Gateway

Enterprise API gateway extended to route, govern, and observe LLM and agent traffic across providers.

Freemium· Free trial: $0 · Plus: Charged per Gateway per month · Enterprise: Custom pricing · Essentials: $0 · Pro: $12AgentsMulti-model7.3 / 10

Visit website →

In short

Kong AI Gateway extends the Kong API gateway to manage LLM and agent traffic, providing a unified control plane for routing, security, and observability. It is best suited for platform and security teams already using Kong who need enterprise-grade governance for multi-LLM workloads.

Best for

Pick Kong AI Gateway if you already run Kong or need an enterprise-grade proxy that governs LLM, MCP, and agent traffic in one place.

Skip if

Skip it if you just need a lightweight SDK to fan out to a few LLM providers — LiteLLM or Portkey are far less to operate.

Kong AI Gateway is the AI-traffic layer built on top of Kong Gateway, the widely deployed open-source API gateway. It puts a single control plane in front of OpenAI, Anthropic, Azure OpenAI, Bedrock, Mistral, Cohere, local models, and MCP servers, exposing a unified API while handling auth, rate-limiting, semantic caching, PII sanitization, prompt guards, token-based quotas, and per-model usage analytics. Multi-LLM routing lets you fail over or A/B between providers without changing application code.

It is aimed at platform and security teams running production LLM workloads who already use (or are willing to run) Kong's data-plane proxy. Pricing follows Kong's standard model: the OSS Gateway plus AI plugins is free to self-host, Kong Konnect adds a managed control plane on a usage-based plan, and Kong Enterprise is quote-only with SSO, RBAC, FIPS, and support SLAs. The install page covers Linux packages, Docker, and Kubernetes via Helm or the Kong Operator.

If you are already running Kong for north-south API traffic, layering on the AI plugins is the path of least resistance. If you are not, the operational cost of standing up a gateway just for LLM governance is real — lighter SDK-only options like LiteLLM or Portkey may be a better starting point. Kong's advantage shows up at scale: multi-team governance, MCP server management, agent-to-agent observability, and integration with the broader API platform.

Editor's take

Kong's bet is that LLM traffic is just API traffic with weirder failure modes, and they're largely right. For platform teams already standardizing on Kong, the AI plugins are the obvious choice. For a two-person startup, it's a lot of YAML for a problem an SDK can solve.

— The AI Tool Bible editorial team

Pros

✅ Unified proxy for 20+ LLM providers with drop-in OpenAI-compatible API
✅ Mature gateway runtime with rate-limiting, caching, auth, and observability baked in
✅ Self-hostable OSS core; no hard dependency on a SaaS control plane
✅ First-class MCP and agent-to-agent traffic management
✅ Plugs into existing Kong API governance and SSO/RBAC

Cons

⚠️ Operational overhead of running Kong is overkill for small teams
⚠️ Enterprise features (FIPS, advanced analytics) gated behind sales
⚠️ Documentation spread across konghq.com, docs, and developer portal

Use cases

llm-gatewaymulti-llm-routingai-observabilityprompt-guardrailsmcp-governancetoken-quotas

Frequently asked

Which LLM providers and protocols does Kong AI Gateway support?: It supports a unified API for OpenAI, Anthropic, Azure OpenAI, Bedrock, Mistral, Cohere, local models, and MCP servers, allowing multi-LLM routing and failover without changing application code.
What security and governance features are included?: The gateway handles authentication, rate-limiting, semantic caching, PII sanitization, prompt guards, token-based quotas, and per-model usage analytics.
Is Kong AI Gateway open source and self-hostable?: Yes, the OSS Gateway plus AI plugins is free to self-host on Linux, Docker, or Kubernetes. A managed control plane (Kong Konnect) and Enterprise features like SSO and RBAC are available via paid plans.
Who is the ideal user for this tool?: It is best for platform and security teams running production LLM workloads who already use Kong or need an enterprise-grade proxy for multi-team governance and agent-to-agent observability.