IndexMapAnti-DoomscrollAbout
ConceptLLMImageCross-Modal

Computer Use

Also known as: Computer-Using Agents · GUI agents · Browser agents

Plain English

Computer use lets an AI model work through the visible controls of a browser or desktop application. It can read what appears on screen and ask a tool to click, type, or scroll.

Technical Overview

A computer-use harness presents screenshots or structured interface observations to the model, executes authorized actions, and returns updated state. This supports software that lacks a suitable API, but changing layouts, ambiguous visual state, and multi-step errors can reduce reliability. Human confirmations and isolated environments help control consequential actions and access.

Timeline (4)

RSS
  1. 2024

  2. 2025

    Release

    OpenAI introduced Operator as a research preview of a model trained to act on visual observations. Its system card described safeguards and evaluated end-to-end task completion in simulated environments.

  3. 2026

    Release

    OpenAI introduced native computer use in its general-purpose GPT-5.4 model. A single model could reason over a task and produce mouse or keyboard actions from screenshots.

See also (7)

Open full graph
2 more connections