Hugging Face moved language-model routing to OpenRouter. A caller using automatic routing can keep the same client API while the router chooses a serving provider based on price, speed, and reliability and retries eligible failures with another provider.
Inference Provider
Also known as: Inference Providers · Hosted Inference
Plain English
An inference provider runs a trained model and returns the result through an API. It handles the machines and capacity needed to answer a prompt, generate an image, or process a video.
Technical Overview
A provider operates the inference engine and manages deployment, scaling, queues, and request limits. A shared endpoint uses capacity across customers, while a reserved or dedicated option commits capacity for a deployment. A router can send a request to one of several providers and retry elsewhere on failure. Latency, throughput, supported model versions, and pricing depend on the provider and deployment.
Chronological Entries (4)
Hugging Face added an interface for choosing which provider runs a model from the Hub. The initial integration included fal.ai and Together AI, allowing a model hosted by a participating provider to be called from the Hub's client libraries.
AWS made Amazon Bedrock generally available, exposing foundation models from Anthropic, AI21 Labs, Cohere, Stability AI, and Amazon behind a single managed serverless API so applications could switch models without operating GPU clusters.
AWS made SageMaker Serverless Inference generally available. The endpoint could provision compute on request and turn it off when idle, allowing an application with intermittent traffic to pay for request processing without keeping its own server running.