Brain Wave Data Emerges as the Next Frontier for Training Robots

In a warehouse in San Leandro, California, a man named Andrew Ceja is carefully pulling wooden blocks from a tottering Jenga tower. He wears a headset with a camera that tracks what he sees, plus sensors that measure his brain waves 1. Ceja is a "pilot," the term Encord uses for its robotic trainers, and every block he pulls generates data for physical AI 1.

The headset was built by Zander Labs, a German neuroscience startup that measures brain activity to deduce mental states like error, intent, and surprise 1. The premise is that those neural signals could automatically label training data for robots. The signals would capture what an operator intends to do, when they slip up, and what catches them off guard, all without a second human manually annotating each frame.

The bottleneck holding back physical AI is not compute or model architecture. It is the scarcity of real-world manipulation data. Large language models were trained on the text of the entire internet 1. Scraping it cost frontier labs next to nothing. Teaching a robot to plug in an ethernet cable or pour coffee without spilling requires something entirely different: data that has to be physically manufactured, frame by frame, action by action 1.

Vineeth Velmurugan leads Encord's robot learning team. A veteran of OpenAI's robot lab and Berkshire Grey, the warehouse automation firm, he estimates that breaking through the data barrier will require a dataset roughly five times the size of YouTube's entire video corpus 1. That scale explains why data generation has become a business rather than a research problem.

Encord was founded to help companies building machine-vision applications annotate data and evaluate models 1. When its customers started applying end-to-end learning to robotic manipulation tasks, the company realized it would need to produce training data itself, not just organize what already existed. Velmurugan says Encord works with many leading robotics firms but is not authorized to name them 1.

The brain-wave experiment is, by Encord's own description, a trial run. The plan is to build an initial brain-wave-tagged dataset, run it through customer robotics models, and evaluate whether it actually improves performance before deciding whether to scale up 1. Lucas Gehrke, a Zander neuroscientist supervising the work, says the amount of brain activity registered at any point during a task offers clues about when model builders need to deploy their highest-effort models 1.

Zander Labs describes its technology as using passive brain-computer interfaces that operate in the background to tap into human mental states, with a stated goal of advancing "the training of the next generation of human-compatible AI systems" 2. The company is developing the Zypher Suite, a wearable EEG hardware system designed to monitor brain activity with local data processing for privacy 2.

Brain waves are one piece of a broader data manufacturing operation. At the San Leandro facility, pilots use paired robotic arms, one controlled by a human operator and one that mimics its movements, to create data for tasks like pouring coffee and stacking poker chips 1. Storage racks hold cartons of fake flowers, books, plastic vegetables, and kitty litter trays, the raw material for training robots to handle household objects 1. Another station uses sensors strapped to the forearm to detect electrical signals in muscles, aimed at building a 3D model of hand position that video alone cannot capture 1.

Every video gets annotated with physical descriptions of the action, such as "right hand tightens bolt" 1. Velmurugan estimates this kind of dense annotation is worth roughly 100 times as much as low-quality egocentric video for training specific tasks, and costs about 20 times more to produce 1. That ratio looks favorable. But 20 times more is still real money, and unlike text scraped from the web, there is no shortcut to generating it cheaply. Ceja and fellow pilot Sofia Infante both previously worked at Scale, another AI data annotation firm, part of a growing workforce assembling the building blocks for physical AI one task at a time 1.

Brain-wave tagging offers a theoretical path through that cost wall. Instead of a human manually labeling each frame, a pilot's neural response could tag the data as the task unfolds. If EEG signals can reliably mark moments of error, intent, and surprise, a labeling step that takes hours could collapse into the time it takes a person to perform the task once. Zander Labs' passive measurement approach means the operator does not need to consciously think about tagging 2.

Whether it works is an open question. The EEG signal from a wearable headset is noisy, and deducing specific mental states from brain activity remains an active area of neuroscience research, not a settled technique. A pilot who reacts to a toppling Jenga tower could be responding to the game, the task, the headset, or the room temperature. Encord has not yet run brain-wave-tagged data through a customer model, and the company describes the effort as an experiment 1.

Encord's vantage point, sitting between many robotics firms at once, means it can spot which data techniques gain traction across the industry before any single customer can 1. If brain-wave tagging compresses the annotation loop, the companies racing to ship humanoid robots gain a shortcut around the field's most expensive bottleneck. If the signal proves too noisy to be useful, the work continues the hard way: someone still has to manufacture millions of hours of physical training data, one Jenga block at a time.