Are Brain Waves the Next Unlock for Physical AI?

Brain Waves and Physical AI

The frontier of physical AI is a Jenga game in a warehouse in San Leandro, California. That warehouse is occupied by Encord — a company that builds data tooling used to train AI models. Andrew Ceja is a pilot, the company's term for its robotic trainers, and he's carefully pulling wooden blocks from a tottering tower while wearing a headset with a camera that tracks what he sees. That alone is fairly common for collecting robot training data, but this headset includes sensors that measure his brain waves as he carefully disassembles the block tower.

TechCrunch's Tim Fernholz visited this very facility to witness firsthand how a growing number of startups are betting that the next real constraint on humanoid and warehouse robotics won't be model architecture — but instead the sheer scarcity of real-world physical training data.

The Data Bottleneck in Physical AI

Physical AI refers to robots and autonomous systems that can act in the real world — manipulating objects, navigating spaces, performing physical tasks. Unlike language models like ChatGPT, which can learn from text scraped from the internet at near-zero cost, physical AI requires embodied experience. Robots need thousands, even millions, of repetitions of simple tasks: picking up a cup, pouring coffee, opening a door. Someone has to demonstrate these actions.

Vineeth Velmurugan, Encord's head of robot learning and a veteran of OpenAI's robot lab and Berkshire Grey, puts the challenge in stark terms: "The data simply does not exist." He estimates it will take a dataset roughly five times the size of YouTube's video corpus to break through — a scale that explains why data generation has become a business, not just a research problem.

Encord's Approach: Manufacturing Data, Not Just Managing It

Encord was originally founded to help companies building machine-vision applications annotate data and evaluate models. But as their customers — leading robotics firms — began applying end-to-end learning to robotic manipulation tasks, executives realized they would have to produce training data themselves, rather than simply manage it.

Today, Encord's San Leandro warehouse is a manufacturing floor for physical AI data. Multiple robotic stations are set up for different tasks: one where robotic arms pour coffee into mugs (very sloshy), another stacking poker chips, a third plugging and unplugging ethernet cables from servers. "Every humanoid company has asked us for these pieces," Velmurugan says.

The facility is stocked with the props of everyday manipulation: boxes of fake flowers in vases, plastic vegetables, books, kitty litter trays and scoops, bags and bundles of wires — all the stock-in-trade for training manipulators for household and industrial tasks. Every action is recorded from multiple camera angles with dense annotation, creating what Velmurugan calls "manufactured" rather than just "collected" data.

Brain-Computer Interfaces: A New Dimension of Training Data

Encord's most intriguing experiment is a partnership with Zander Labs, a German neuroscience startup that's betting that measuring brain activity during physical tasks can create a far more useful dataset for training models. The headset Ceja wears while playing Jenga includes EEG sensors that capture neural signals in real time as he performs each action.

The concept is deceptively simple: when a human performs a physical task, their brain produces signals that encode intent, error detection, and surprise. If a robot can learn to interpret these neural signals during training, it gains a deeper understanding not just of what happened, but of what the human was trying to do.

Lucas Gehrke, a Zander neuroscientist supervising the work, explains that the amount of brain activity at any point during a task offers clues for model builders trying to figure out when they need to deploy their highest-effort models. A sudden spike in neural activity might indicate the human detected an error or is about to adjust their approach — precisely the kind of information a robot needs to learn robust manipulation.

This is, according to Velmurugan, the "bleeding edge" of the effort to solve the robotics data bottleneck. Encord's work with Zander is currently a trial run: the goal is to build an initial brain wave-tagged dataset, run it through customer robotics models, and evaluate whether it actually improves performance before deciding whether to scale.

Muscle Signals and Multi-Modal Data

Brain waves aren't the only new modality Encord is exploring. Another experimental approach uses sensors strapped to the forearm to detect electrical signals in muscles. Video footage of human hands manipulating objects typically doesn't capture the entire hand — fingers get hidden behind objects. But Velmurugan hopes to build a 3D depiction of where the hand is at any time based on arm sensors, creating a more robust understanding for models.

Encord's datasets are annotated with physical descriptions of what each video contains — "right hand tightens bolt" — to aid LLM-based models in understanding what is happening. Velmurugan estimates this kind of dense annotation is worth 100 times as much as "junky ego data" for training specific tasks, yet it only costs 20 times more to produce — a good trade-off on paper.

The New Workforce: Robot Training Pilots

Both Andrew Ceja and Sofia Infante, the pilots TechCrunch met at Encord's facility, are part of a burgeoning workforce developing the building blocks for neural networks. They previously worked at Scale, another AI data annotation firm. Ceja had worked at a waste management company where his interest in technology found him in charge of keeping a robotic trash sorter in good working order.

Now, as the Jenga tower topples, he says he enjoys the challenge of solving training tasks for robots. "It's something new every day!" This emerging profession — part demonstrator, part data curator — represents a new category of human work in the AI age.

The Economics Gap: LLMs vs. Physical AI

Despite the promise, Encord's approach highlights a fundamental difference between language AI and physical AI: cost. Frontier labs built large language models by scraping text off the internet — from Stack Overflow, Wikipedia, and the rest of the web — for nearly zero cost. Physical training data must be manufactured, not just collected, and that changes the economics of building these models dramatically.

But Velmurugan remains optimistic. With Encord's visibility into programs across the industry, he's able to see startups and frontier labs alike figure out what works and what doesn't to improve physical AI models. That vantage point — sitting between many robotics companies at once — is also part of Encord's pitch: they can spot which data techniques are gaining traction industry-wide before any single customer can.

What This Means for the Future of AI Training

Encord's experiments with brain waves, muscle signals, multi-camera video, and dense annotation point to where physical AI is heading. The future isn't just about building better models — it's about building better, richer, more precise datasets. If brain waves do prove to improve robot learning, it could be a breakthrough moment for humanoid robotics.

Today's humanoid robots still struggle with tasks humans find trivial — plugging in a cable, pouring from a cup, picking a single item from a cluttered shelf. Tomorrow's robots, trained on multi-modal neural data, might perform thousands of complex actions with human-like dexterity.

And it all starts in a warehouse in San Leandro, where someone wearing an EEG headset carefully pulls a wooden block from a Jenga tower. It may not look like the future of AI — but that's exactly what it is.

📖 Source