Inference Aggregators, Gateways & Routers
26 companies
0 of 0 categories open
Relationship:
Loading the map…
On this page
Services that sit between an application and one or more model providers, offering a single API across many models plus some combination of routing, caching, failover, rate limiting and spend tracking. Several run inference themselves. That is why companies here turn up in the GPU cloud category too.
What to compare
Compare the model catalogue, whether the service hosts models or brokers out to others, the latency it adds, and what happens when an upstream provider falls over. Billing shapes vary. Some resell provider pricing at a margin, others charge a flat gateway fee on top of your own provider keys.
26 companies in this category
AI/ML APIAggregated API access to hundreds of AI models under one key.
AnyscaleManaged Ray platform for scaling AI and Python workloads, including LLM serving.
BasetenPlatform for deploying and serving custom ML models in production.- CentMLInference optimization platform for reducing LLM serving costs.
Cloudflare Workers AIServerless inference running on Cloudflare's global edge network.
DeepInfraLow-cost hosted inference for open-source models.
Featherless AISubscription API giving unlimited access to a large catalog of open models.- Fireworks AIFast, low-cost inference platform for open and custom models.
GMI CloudGPU cloud provider focused on AI training and inference workloads.
GroqCustom LPU chip maker offering extremely low-latency LLM inference.
HeliconeOpen-source LLM observability platform that also proxies requests.
HyperbolicDecentralized GPU cloud and inference marketplace.
LangDBAI gateway with governance, routing, and observability for LLM traffic.
LiteLLMOpen-source library providing a unified API across LLM providers.
MartianModel router that dynamically picks the best LLM per request.
NLP CloudHosted API for open-source and custom NLP/LLM models.
Nebius AI StudioManaged model API platform from GPU cloud provider Nebius.
Novita AIGPU cloud and model API platform for generative AI workloads.
OpenRouterUnified API routing requests across dozens of LLM providers.
PortkeyAI gateway for routing, caching, and observability across LLM providers.
ReplicatePlatform to run and host open-source ML models via a simple API.
SambaNova CloudFast hosted inference on SambaNova's custom AI chips.
Together AIInference and fine-tuning platform for open-weight models at scale.
Unify AIBenchmark-driven router for choosing the optimal LLM/provider per query.
Vast.aiMarketplace for renting GPU compute at low cost.
Vercel AI GatewayUnified gateway for routing AI requests across providers from Vercel.