LiteLLM vs agentgateway vs skilder: three layers, not three rivals

TL;DR

LiteLLM, agentgateway and skilder show up in the same architecture reviews, and they are rarely alternatives to each other. LiteLLM is a model gateway: it puts one OpenAI-compatible endpoint in front of 100+ LLM providers and meters every call. agentgateway is an agent and tool gateway: it proxies MCP, A2A and LLM traffic and applies policy to each request. skilder is the role layer: it defines what an agent may do for one seat on the org chart, as skills plus scoped tools plus permissions, and serves that on demand. The useful question is not “which one” but “which one first, and what does it not solve”.

The question behind the comparison

A platform team evaluating agents in 2026 meets three product categories with overlapping vocabulary. All three talk about governance, observability and MCP. All three publish a diagram with an agent on the left and company systems on the right. Only one thing separates them cleanly: the unit each one controls. A model call, a tool call, or a job.

This piece takes the three names most often written on the same whiteboard and sorts them by that unit. Facts about LiteLLM and agentgateway come from their own documentation and repositories, linked as they appear.

Layer 1: the model gateway (LiteLLM)

LiteLLM describes itself as “an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers”. It ships in two forms: a Python SDK for direct integration, and a proxy server, the part most teams mean when they say “the gateway”. Applications call the proxy in the OpenAI format; the proxy translates to OpenAI, Anthropic, Gemini, Bedrock, Azure and the rest.

The proxy’s feature list, per the LiteLLM proxy documentation, is the feature list of a model gateway: virtual keys, spend tracking with “budgets + rate limits” per key or user, “load balancing, routing, fallbacks (failover)” across deployments, guardrails on requests and responses, and “logging, alerting, metrics” through integrations such as Langfuse and MLflow. Deployment is pip, Docker or Helm; configuration is a config.yaml listing models and their credentials.

Licensing: the repository is MIT licensed, with one carve-out in the licence text: content under the enterprise/ directory is covered by a separate licence. SSO and some admin features sit behind that commercial licence, per the README.

The unit of control is the model call. LiteLLM decides which provider answers, at what cost, under which key, and records the result. It does not know which tool the agent will call next, and it does not know what job the person was doing.

Layer 2: the agent and tool gateway (agentgateway)

agentgateway calls itself a “next generation agentic proxy for AI agents and MCP servers”, offering “drop-in security, observability, and governance for agent-to-LLM, agent-to-tool, and agent-to-agent communication”. It is written in Rust, licensed under Apache 2.0, and hosted by the Linux Foundation after solo.io donated the project.

Where LiteLLM starts from the model, agentgateway starts from the protocols an agent speaks. The README lists an MCP gateway with “tool federation, stdio/HTTP/SSE/Streamable HTTP transports”, an A2A gateway for “secure agent-to-agent communication”, and an LLM gateway with “budget and spend controls, prompt enrichment, load balancing, and failover”. Policy is enforced with “auth (JWT, API keys, OAuth), fine-grained RBAC with CEL policy engine, rate limiting”, and telemetry is OpenTelemetry metrics, logs and traces. It runs as a standalone binary driven by YAML, in Docker, or on Kubernetes with a built-in controller and Gateway API support, per the deployment docs.

The unit of control is the tool call (and the agent-to-agent call). agentgateway can allow one tool on an MCP server and deny another, multiplex several servers behind one endpoint, and log which identity called which tool. It is the enforcement point an MCP registry needs so that “approved” means something at runtime. It does not decide which tools a given job needs, and it does not carry the instructions that make an agent good at that job.

Layer 3: the role and context layer (skilder)

A role, in skilder’s vocabulary, is the work one seat on the org chart actually does, assembled from skills in the open Agent Skill format (SKILL.md files), scoped MCP tools drawn from an internal, curated server registry, the context those need, and the permissions that bound them. A role is configured from a blueprint to a team’s own processes, then served over MCP to whatever agent the person already uses: Claude, ChatGPT, Copilot or an in-house client.

skilder is not a gateway. It does not sit between the agent and the model on every token, and it does not proxy raw MCP traffic. It sits one level up and answers a question neither gateway asks: what is this agent allowed and equipped to do, for this job, right now? The answer is the role. Its skills and tools are loaded on demand, at the moment the task needs them, rather than dumped into the context window at session start, which is the difference the context bloat problem turns on.

Governance in skilder is governance of the role: versioned and signed packages, promotion and rollback organisation-wide, role-based access, an execution trace attributable to the person who ran the task, and usage that shows which roles are used, by whom and how often. Model choice is routed per task across 100+ models, so the role does not change when the provider does.

The unit of control is the job. That is also the boundary of what skilder does: it does not meter tokens per provider the way LiteLLM does, and it does not enforce network policy on arbitrary MCP traffic the way agentgateway does.

Side by side

	LiteLLM	agentgateway	skilder
What it governs	Model calls (source)	Tool, agent and model calls in transit (source)	What an agent may do for one job: skills, scoped tools, permissions (source)
Unit of control	Request to a model, per virtual key	MCP, A2A or LLM request, per identity and policy	Role, per person and team
Protocol surface	OpenAI-compatible API in, 100+ providers out; MCP server registry per key and team (source)	MCP (stdio, HTTP, SSE, Streamable HTTP), A2A, OpenAI-compatible LLM routing	Roles exposed as MCP servers to any MCP client; skills in SKILL.md
Who configures it	Platform or ML team, via config.yaml and the admin UI	Platform or network team, via YAML or Kubernetes Gateway API	Builders and domain experts, from a blueprint in a visual interface
Licence	MIT, with a separate licence for the enterprise/ directory (source)	Apache 2.0 (source)	Multi-tenant SaaS, private deployment on request
Where it runs	Self-hosted: pip, Docker, Helm	Self-hosted: binary, Docker, Kubernetes	Hosted in Switzerland and the EU by default; on-premise on request

Where they overlap, honestly

The three are not disjoint, and the overlaps are where a review gets confused.

LiteLLM reaches into tools. Its MCP gateway lets an administrator register MCP servers once, “use a fixed endpoint for all MCP tools and control MCP access by key, team”, and expose those tools through the /v1/chat/completions and /v1/responses endpoints across every supported model. If the tools an agent needs are few and the agent is an application the platform team writes, that may be enough tool governance.

agentgateway reaches into models. Its LLM gateway handles budgets, spend and failover across providers. If the traffic that matters is MCP and A2A, adding model routing in the same data plane avoids a second proxy.

skilder routes models and scopes tools, but per role. A role can only reach MCP servers already in its workspace registry, and a task is routed to a model chosen for cost, latency or sensitivity. That is policy at the level of the job, not of the network. It complements a gateway; it does not replace the packet-level enforcement a gateway gives.

The distinction that holds up: a gateway decides whether a call passes. A role decides whether the call is part of the job at all, and carries the know-how that makes the call worth making.

Which one do you need first

Start from the question your organisation is asking this quarter.

1. “Our model spend is spread over five providers and nobody can attribute it.” A model gateway. LiteLLM’s virtual keys and budgets answer this directly, and the application side barely changes because the interface stays OpenAI-shaped.
2. “Employees and internal agents are connecting MCP servers nobody reviewed.” An agent gateway, in front of an internal registry. That is the shadow MCP problem, and it needs a runtime enforcement point, not a spreadsheet.
3. “Nobody can say what an agent is allowed to do for a given job, or why it does the job differently for each person.” A role layer. This is the question that stays open after both gateways are in place, because a gateway enforces rules on calls and says nothing about the task. It is also where the company’s know-how lives, and where skills over MCP tools turn a governed setup into a usable one.

Two of the three is the common answer. A model gateway plus a role layer covers a company whose agents are the vendor clients (Claude, ChatGPT, Copilot) and whose tools come through a curated registry. An agent gateway plus a role layer covers a company running its own agents on Kubernetes with MCP traffic to police. All three is reasonable at scale, and redundant before it.

The bottom line

LiteLLM meters the model. agentgateway polices the wire. skilder defines the job. Confusing the three costs a quarter of evaluation; stacking them in the right order costs a config file each.

To see what a role looks like in practice, configure a first one from a blueprint at app.skilder.ai/signup, or read how a role is exposed to an MCP client in the documentation.

Frequently asked questions

Is skilder a replacement for LiteLLM or agentgateway? +

No. LiteLLM and agentgateway sit in the request path and enforce policy on model calls and tool calls. skilder sits above them: it defines roles (skills plus scoped MCP tools plus permissions) and serves them to the agent. A role's tool calls can flow through agentgateway, and skilder's model routing can point at a LiteLLM endpoint. The layers complement each other.

Do LiteLLM and agentgateway overlap? +

Partly. LiteLLM documents an MCP gateway that registers MCP servers and controls their access by key, team and organisation; agentgateway documents LLM routing with budgets and failover. Each started on one side (models for LiteLLM, MCP and A2A for agentgateway) and grew toward the other. Pick by the traffic that matters most to you today, and by the deployment model your platform team already runs.

Which one should a company deploy first? +

If the pressing question is model spend and provider sprawl, a model gateway. If it is unvetted tools reaching company systems, an agent gateway. If it is that nobody can say what an agent is allowed to do for a given job, a role layer. In practice the third question arrives the moment the first two are answered, because a gateway enforces rules on calls but does not say what the job is.