FIFoundationIndex
IndexMapAbout
ConceptCross-ModalLLMImageVideo

Evals & Benchmarks

Also known as: Evals · Chatbot Arena · SWE-bench · LLM-as-a-Judge

Plain English

The test suites and leaderboards that measure whether an AI model—or an AI application you built—is actually getting better or silently regressing across accuracy, coding, visual realism, safety, and latency.

Technical Overview

Evaluation divides into (1) public capability benchmarks—coding (SWE-bench Verified), math/reasoning (AIME, GPQA, ARC-AGI), human-preference Elo arenas (LMArena / Artificial Analysis Image & Video Arenas)—and (2) task-specific application evals using deterministic assertions, citation-faithfulness checks, and calibrated LLM-as-a-Judge rubrics over golden datasets.

Chronological Entries (2)