IndexMapAbout
ConceptLLMImageVideoCross-Modal

Inference Provider

Also known as: Inference Providers · Hosted Inference

Plain English

An inference provider runs a trained model and returns the result through an API. It handles the machines and capacity needed to answer a prompt, generate an image, or process a video.

Technical Overview

A provider operates the inference engine and manages deployment, scaling, queues, and request limits. A shared endpoint uses capacity across customers, while a reserved or dedicated option commits capacity for a deployment. A router can send a request to one of several providers and retry elsewhere on failure. Latency, throughput, supported model versions, and pricing depend on the provider and deployment.

Chronological Entries (4)

Direct Relations (1)