AI learns from watching screens without labels—Photon-1 shows how
An AI agent that can run a desktop, play checkers or model billiard physics without ever seeing an action label has quietly outperformed a mainstream model while costing far less. Induction Labs’ Photon-1, a 106B-A5B sparse mixture-of-experts transformer, learned from 18 years of raw screen recordings—no clicks, keystrokes or rewards supplied during pretraining. On an internal benchmark of computer use, it surpassed Google’s Gemini 3.1 Flash-Lite despite using roughly 27× less pretraining compute and serving at about one-third the cost.
Learning from pixels, not prompts
Photon-1 belongs to a new class called imagination models. Instead of mapping frames to actions, it learns to predict future latent states autoregressively. The trick is a differential latent encoder that condenses each frame into 960 discrete tokens—about 2.2 KB—using finite scalar quantization. By encoding frame-to-frame differences, the model captures text, layout and state changes with over 100× compression versus traditional OCR or multimodal encoders. Crucially, this pretraining objective teaches an “implicit policy”: the model infers what a person is doing without ever seeing a labeled action.
Sparsity and scale in practice
The architecture mixes 106 billion parameters with only 5 billion active at each step, trained on 552 billion tokens from 575 million keyframes sampled at 1 frame per second. Induction Labs ran a single epoch on 30,000 H200 GPU-hours, sustaining 40% model-flops utilization and logging roughly 4.4×10²² FLOPs. After pretraining, the team fine-tuned Photon-1 on fewer than 35,000 computer-use trajectories to add action and instruction formats. At inference, the model first predicts the next frame’s latent state, then emits the action that would reach it; online reinforcement learning verifies outcomes on real Linux VMs running five desktop environments.
Why it matters
If validated beyond benchmarks, imagination models could lower the barrier to training AI agents that operate digital interfaces. By removing the need for action-labeled videos, teams gain flexibility and cut costs dramatically. The real test will be whether these implicit policies transfer to broader, unseen tasks and real-world latency constraints. For developers and researchers, the advance signals a shift from supervised action modeling to self-supervised state prediction—a move that may reshape how we build generalist desktop agents.
Source: MarkTechPost. AI-assisted editorial synthesis — TechnoExpress.