The Chatbot Arena team introduced anonymous, randomized model battles and published an initial Elo leaderboard. Visitors could submit the same prompt to two hidden models and vote on the answers.
Arena (LMArena)
Also known as: Chatbot Arena · LM Arena · Arena.ai
Plain English
Arena compares AI models by showing people two anonymous answers and asking them to vote. It aggregates comparisons into leaderboards for chat and other tasks.
Technical Overview
Arena draws on pairwise preference evaluation across user-submitted prompts. Statistical models estimate relative rankings from votes and use uncertainty intervals to show when systems are hard to distinguish. Scores reflect the prompts, users, and model configurations sampled by each arena, including the tools available in some tracks.
Timeline (4)
2023
Release2024
PaperThe Chatbot Arena paper described its pairwise evaluation and statistical ranking methods. The authors compared crowdsourced judgments with expert ones and studied whether the prompts separated models across capabilities.
2025
EcosystemThe Arena team documented Search Arena, a model comparison for search-enabled systems. The track observes how people judge answers when both the generated text and the retrieved sources affect usefulness.
2026
EcosystemArena described Agent Arena, a comparison using real user tasks with tools such as web search and code execution. Its methodology reported costs and execution behavior alongside user preferences.