FIFoundationIndex
IndexMapAbout

Foundation Model Ecosystem Map

40 subjects across 6 architectural & ecosystem layers connected by 317 cross-links and typed relations.

Modality:
Kind:
1

Core Architectures & Math

Foundational neural building blocks for representing, attending to, and generating text, images, and video.

8 subjects

Diffusion Transformer (DiT)

An architecture that swaps out the older convolutional U-Net inside image and video generators for a Transformer—bringing the same massive scaling laws that powered LLMs into visual generation.

Architecture
222e

Diffusion Models

The dominant family of generative models for images and video. They learn to create realistic visuals by starting from pure random noise and gradually sculpting it into a coherent picture over multiple steps.

Architecture
193e

Transformer

The core neural network architecture behind modern LLMs and increasingly image and video generators. Instead of reading data strictly step-by-step like older recurrent networks, a Transformer compares every piece of an input sequence against every other piece in parallel.

Architecture
193e

KV Cache

A short-term memory buffer stored in GPU VRAM while an LLM generates text. Without it, the model would have to re-read the entire conversation from scratch for every single new word it outputs.

Concept
173e

Multimodal & Vision-Language Models

AI models that understand—and increasingly generate—multiple types of media together, such as text, images, screenshots, audio, and video, inside one shared model.

Architecture
132e

Latent Space & VAE

A learned compression layer that shrinks high-resolution images or videos into a compact numerical map (latent space) so generative models can work faster, then expands the result back into crisp pixels.

Concept
122e

Tokenization

How AI models chop text (and increasingly audio or image patches) into numerical chunks called tokens before processing them. A token is roughly three-quarters of an English word, which is also how LLM APIs measure pricing and speed.

Concept
92e

Mixture of Experts (MoE)

A design that packs multiple specialized sub-networks ('experts') inside one large model, but only activates a small fraction of them for any given token—delivering big-model intelligence at small-model speed.

Architecture
92e
2

Weights, Formats & Adaptation

How model parameters are packaged, quantized, fine-tuned, aligned, and safely distributed.

8 subjects

Checkpoints

A saved snapshot of a neural network's learned numbers (weights) at a specific point in training. Downloading a 'checkpoint' gives you the complete, runnable brain of a model—whether an LLM or an image generator.

Concept
273e

LoRA & QLoRA

A lightweight 'patch' file (often tens of megabytes instead of gigabytes) that teaches an existing model a new style, character, domain, or skill without retraining or duplicating the entire base model.

Concept
203e

Quantization

Shrinking a model's weights from high-precision decimal numbers (like 16-bit floats) down to smaller numbers (8-bit or 4-bit) so the model uses far less GPU memory and runs faster, with minimal loss in quality.

Concept
202e

Safetensors

The modern, safe file format for storing and sharing AI model weights. It replaced older Python pickle files (.ckpt/.bin) that could hide malicious code, and it loads models into GPU memory much faster.

Format / Protocol
182e

GGUF

A single-file model format built for running AI models locally on laptops, Apple Silicon Macs, and consumer GPUs—packaging compressed weights and tokenizer metadata together so you can run a model with zero complex setup.

Format / Protocol
162e

Distillation & Synthetic Data

Using a huge, expensive 'teacher' model to generate high-quality training examples or supervision signals so a much smaller, faster, cheaper 'student' model can punch far above its weight.

Concept
122e

Post-Training (SFT, RLHF, DPO & GRPO)

Everything done to a model *after* initial pre-training to turn a raw text-autocomplete engine into a helpful, safe assistant or a multi-step reasoning model that checks its own work.

Concept
112e

ControlNet & IP-Adapter

Add-on modules that give you precise visual steering over an image or video generator—letting you lock in a pose, architectural layout, depth map, sketch, or reference face/style instead of relying on text prompts alone.

Architecture
72e
3

Inference, Serving & Controls

Systems engineering, GPU kernels, memory optimization, and sampling controls for running models fast.

6 subjects

Inference Engines (vLLM, SGLang, llama.cpp)

Specialized runtime servers that actually execute AI models in production or on your laptop—squeezing 5–20× more speed and concurrent users out of the same GPU compared to naive Python scripts.

Tool / Runtime
212e

Context Window & RoPE

The working memory limit of a model—how many tokens of text, code, or video frames it can 'see' at once—together with RoPE, the math trick that lets models know which token came before which even across hundreds of thousands of tokens.

Concept
182e

Classifier-Free Guidance (CFG)

The 'prompt adherence slider' in image and video generators. Turning CFG up forces the model to follow your text prompt more strictly—and enables negative prompts—while turning it too high oversaturates or 'burns' the image.

Concept
112e

FlashAttention & GPU Kernels

Low-level GPU engineering breakthroughs that compute Transformer attention 2–8× faster and with far less memory—without changing the mathematical result—by keeping data inside ultra-fast on-chip SRAM instead of constantly shuttling it to main VRAM.

Concept
102e

Sampling & Schedulers

The math routines that turn raw model predictions into actual outputs—controlling creativity vs. determinism in LLMs (temperature, top-p) and controlling how noise is stepped down into pixels in image and video models (Euler, DPM++, FlowMatch).

Concept
92e

Speculative Decoding

A lossless speedup trick for LLMs: a tiny fast 'draft' model guesses the next several words ahead, and the big smart model verifies all of them in a single parallel check—generating 2–3× faster with mathematically identical output quality.

Concept
62e
4

Labs, Companies & Hubs

Research labs, open-weight ecosystems, and distribution platforms shaping the foundation model landscape.

8 subjects

Hugging Face

The 'GitHub of AI'—the central open platform where researchers, labs, and developers host, discover, and download open-weight model checkpoints, datasets, demos (Spaces), and core Python libraries.

Company / Hub
252e

OpenAI

The AI research lab behind ChatGPT, the GPT model lineage, reasoning models (o-series), and the Sora video generator. Its API design also became the unofficial HTTP standard copied by almost every open-source serving engine.

Company / Hub
212e

DeepSeek

A Chinese AI research lab that shook the global AI industry by releasing MIT-licensed open-weight models matching top proprietary labs at a fraction of reported training and inference costs, while publishing full architectural blueprints.

Company / Hub
212e

Black Forest Labs

The German AI lab founded by the original creators of Latent Diffusion / Stable Diffusion. They build the FLUX family of image models, which quickly became the gold standard for open-weight photorealism and typography.

Company / Hub
162e

Google DeepMind

Google's flagship AI research organization—home to the Gemini multimodal LLMs, the open-weight Gemma family, Imagen image models, Veo video models, and foundational breakthroughs like the original Transformer and AlphaFold.

Company / Hub
142e

Anthropic

The AI safety and research lab behind the Claude family of models, known for strong software engineering performance, long-context reasoning, mechanistic interpretability research, and creating the Model Context Protocol (MCP).

Company / Hub
142e

Meta AI

Meta's AI research and product group (including FAIR), famous for kickstarting the open-weight LLM ecosystem with the Llama family as well as foundational open-source tools like PyTorch and Segment Anything (SAM).

Company / Hub
132e

Runway

One of the earliest and most influential AI video labs—co-creator of the original Stable Diffusion research and builder of the Gen-2, Gen-3, and Gen-4 video generation models geared toward filmmakers and studios.

Company / Hub
62e
5

Representative Model Families

Landmark open-weight and frontier model lineages across text, image, and video generation.

3 subjects
6

Workflows, Tooling & Agent Stack

Node editors, model marketplaces, inference clouds, retrieval pipelines, protocols, and evaluation harnesses.

7 subjects

ComfyUI

The dominant open-source node-graph studio for image and video AI. Instead of a fixed form with a few sliders, ComfyUI lets you wire together checkpoints, LoRAs, ControlNets, VAEs, and upscalers like modular audio synths—and export the exact graph as JSON.

Tool / Runtime
272e

Serverless GPU & Inference Clouds

Pay-per-use cloud platforms that host open-weight LLMs, image/video models, and custom GPU code behind simple HTTP APIs—so developers don't have to rent idle GPUs or manage Kubernetes GPU clusters themselves.

Company / Hub
192e

Civitai

The largest community hub and gallery specifically for open-source image and video AI assets—where creators share fine-tuned checkpoints, style/character LoRAs, ControlNets, and ComfyUI workflows alongside visual preview examples.

Company / Hub
141e

Evals & Benchmarks

The test suites and leaderboards that measure whether an AI model—or an AI application you built—is actually getting better or silently regressing across accuracy, coding, visual realism, safety, and latency.

Concept
122e

RAG & Hybrid Retrieval

Giving an AI model a live search engine over your own documents, databases, or code before it answers—so responses stay grounded in up-to-date, verifiable facts instead of relying only on frozen training weights.

Concept
112e

Model Context Protocol (MCP)

An open 'USB-C standard' for plugging AI models and agents into external tools, databases, APIs, and local files—so developers write a connector once instead of rebuilding custom integrations for every AI app.

Format / Protocol
92e

LangGraph & Agent Orchestration

A framework for building reliable, multi-step AI workflows as explicit state machines (graphs)—supporting loops, retries, human-in-the-loop approvals, and durable memory rather than fragile one-shot prompts.

Tool / Runtime
82e