Robotic avatars

Robotic avatars are robots that a human operator drives in real time from somewhere else, seeing through the robot's cameras, hearing through its microphones and, in the more capable systems, feeling the forces its hands meet. Susumu Tachi named the idea "telexistence" in 1980, and Marvin Minsky's "telepresence", proposed independently the same year, covers much the same ground.¹ The ANA Avatar XPRIZE, a $10 million competition for such systems, was announced in 2018 and decided in 2022.² What separates an avatar from an ordinary remote-controlled machine is the aim of embodiment: the operator should act through the robot as though it were their own body, rather than steer it from a console.³

§How it works

An avatar system has an operator station, a robot and a network between them. The operator usually wears a head-mounted display fed by stereo cameras on a robot head that follows the operator's own head movements. Operators often get motion sickness while the robot is moving, because its camera image is not stabilized the way the eye's image is, and delay in the video stream can make this worse.⁴ The winning XPRIZE team hid head-motion latency with a technique it calls spherical rendering.⁵

Arm control comes in two broad forms. In task-space control the station tracks only the operator's hand and software computes how the robot's arm should reach it. In joint-space control the operator moves a leader arm whose joints correspond one to one with the robot's, and the follower copies each joint. Leader-follower designs of this kind date back at least to 1953, when they were built to handle hazardous material. The low-cost ALOHA research system uses the same principle, partly because off-the-shelf inverse kinematics failed frequently near singular poses of its six-joint arms.⁶ The XPRIZE winner, NimbRo, gave operators a full replica of the robot's arms, and operators reported that there was no "mapping" to learn.³

Force feedback closes the loop in the other direction. Forces measured at the robot are reflected to the operator's hand or arm, so that a solid object stops the operator's movement. A survey of humanoid teleoperation argues that a real sense of telepresence needs this force coupling, and it is the part most sensitive to delay. People react to visual input in roughly 250-300 ms but to haptic input in roughly 100-150 ms, and round-trip delays of around 100 ms, typical of the internet, can make a force-reflecting system unstable.⁴ Some operator stations use upper-body and hand exoskeletons to read the operator's movements and push back on them.² The device is the inverse of a powered exoskeleton: it adds no strength, and carries information instead. Avatar haptics deliver touch to an intact hand from outside the body, whereas bidirectional Neuroprosthetics try to deliver it by stimulating nerves or the brain.

§Development history

Remote manipulation began at Argonne National Laboratory soon after the Second World War, for handling radioactive material.¹ Tachi describes telexistence as growing out of General Electric's unsuccessful Hardiman exoskeleton, which was potentially dangerous to wear if the machine malfunctioned and could not work autonomously: a "virtual exoskeleton human amplifier" in which the operator feels present in a remote robot while staying out of harm's way.¹ His architecture from the early 1980s already mixed modes, with several robots working autonomously under supervisory control and the operator switching into telexistence when a task exceeded what a robot could do alone.¹

1. 1980
   Telexistence named
   Susumu Tachi conceives telexistence in September 1980; Marvin Minsky independently proposes telepresence the same year.
2. 1983
   Hazardous-environment programme
   Japan begins an eight-year national project on advanced robot technology for hazardous environments, built around the telexistence concept. The first TELESAR master-slave system follows.
3. 2001
   Robonaut
   NASA's 43-degree-of-freedom Robonaut is operated mainly through telepresence, with gloves and a helmet mapping the operator's hands, arms and neck onto the robot.
4. 2011
   A humanoid in orbit
   Robonaut 2, built by NASA and General Motors, launches to the International Space Station on the shuttle Discovery.
5. 2012
   Touch and temperature
   Tachi's TELESAR V transmits fingertip texture and temperature from the robot's fingers to the operator's.
6. 2018
   ANA Avatar XPRIZE
   XPRIZE announces a $10 million competition after its design team visits Tachi's laboratory and sees TELESAR V demonstrated.
7. 2022
   XPRIZE finals
   Seventeen teams compete in Long Beach, California. NimbRo of the University of Bonn wins $5 million, ahead of Pollen Robotics and Team Northeastern.

By Tachi's account, the XPRIZE grew out of a 2016 visit by the foundation's prize-design team to his laboratory, where they tried TELESAR V.¹ NASA's Robonaut came from a different motive. Its designers described spacewalk maintenance as expensive and hazardous, with astronauts pre-breathing at suit pressure for up to four hours before leaving the station, and designed the robot to work in the crew's corridors and use much of the same spacewalk tool set. They reported that novice operators became proficient in under five minutes, and planned to move the operator gradually towards a supervisory role.⁷ Robonaut 2 was launched to the International Space Station on the shuttle Discovery on 24 February 2011.⁸ The effects of spaceflight on crews themselves are covered under Space medicine.

§The ANA Avatar XPRIZE

The competition was run by the foundation also behind XPRIZE Healthspan. Ninety-nine teams from 20 countries registered, and $2 million in semifinal prize money was paid in equal shares to the top 15 teams that tested in Miami in September 2021. The finals ran from 1 to 5 November 2022. Each robot had to run on batteries over the organisers' WiFi, and its operator was not a team member but a judge given 45 minutes of training before a ten-task mission with a 25-minute limit. The tasks included picking out a heavy, full canister from empty ones, using a power drill, and finding a rough stone by touch alone.²

Four teams completed all ten tasks. NimbRo finished in 5 minutes 50 seconds, about 35 seconds per task, which the report's authors judged not much slower than a person doing the tasks directly. The weak points were the senses that video does not carry. At the semifinals, the question on whether operators received the force and touch feedback they needed scored lowest of any experience question, with the top 20 teams earning 54% of its available points. In the finals, the drill succeeded 67% of the time and the texture task 60%. Network connectivity was the largest single cause of failure, and in 32% of runs the robot took a significant time to start moving.²

A joint analysis by organisers, judges and team leads added that complex, human-scale humanoids carried a higher risk of catastrophic failure, that wheeled bases beat walking ones on the flat course, and that video lag of several seconds was often the root cause of failed runs. Teams using ordinary screens rather than headsets showed no statistically significant penalty, though with only 16 teams in the analysis many effects could not be detected.³

How human should an avatar be

The competition report's authors wrote that it remains open how much human-likeness avatars need or should have, while noting that head movement, gaze, gestures and an animated face helped recipients perceive the operator as present.² The joint analysis found that simpler robots often performed reasonably well, perhaps because they were easier to control, and that the more complex humanoid systems were at elevated risk of catastrophic failure.³ Neither report settles how human-like an avatar should be.

§Current state

Research systems now work over real distances. The Italian Institute of Technology's iCub3, a walking humanoid, was operated from Genoa during a visit to the Venice Biennale about 290 km away over ordinary fibre internet, with communication delay staying below 25 ms; it was later operated on stage in Rimini, about 300 km away.⁹ The XPRIZE organisers had dropped an early goal of operating from at least 100 km away, reasoning that distance adds little delay compared with local WiFi and high-definition video streaming. At the finals, operators and robots were in the same building, and the arena WiFi, which did not perform consistently, affected reliability and speed.³

Hazardous work is the oldest motivation. A humanoid teleoperation survey notes that after the Fukushima Daiichi accident it took more than three months to deliver a robot able to perform its first mission inside the plant.⁴ Robots for work on energised power lines have a literature of their own: a teleoperated system for live-line maintenance was described in 1995,¹⁰ and in 2024 the IEEE published a guide setting general technical requirements for live-working robots on 6-35 kV distribution lines.¹¹

Commercial deployments are narrow and mostly reported by the companies involved. In 2022 NVIDIA reported that shelf-restocking robots from Telexistence, which run on NVIDIA hardware, would go to 300 FamilyMart convenience stores in Japan, restocking automatically more than 98% of the time, with remote staff taking manual control through virtual reality when a drink toppled.¹²

Mundane, a Palo Alto company, is one current attempt to aim the full avatar design at the skilled trades, such as linework and other work in energised spaces. By its own account, the operator wears a force-feedback exoskeleton whose joints match the robot's in number, order and axes, so that force is felt along the whole arm rather than only at the hand, and there are "no modes, buttons or mappings to learn"; the page marks the design patent pending. Customers lease the system, their own workers operate it, and "where customers allow it" the work becomes training data. The company calls keeping force feedback stable when the operator is far away "the hardest part, and the part we are working on now". It publishes no performance metrics, and it describes its exoskeleton as combining whole-arm motion capture with the force feedback of leader-follower systems.¹³ It says more than 100 tradespeople operated the robot at NECA 2026 in Las Vegas.¹⁴

What the evidence covers

The best-documented results come from briefly trained judges on a flat competition course, and from a user study of 35 people aged 20-34 in rooms about 30 m apart.⁵ None of the evaluations cited here measures accuracy, error rates or operator fatigue for avatar robots doing real hazardous work over full shifts. Company accounts are unreviewed, and Mundane's comes without performance numbers.

§Teleoperation as training data

The operator's movements are themselves a record of how a skilled task is done, and robot-learning research uses teleoperation to collect demonstrations. ALOHA, a bimanual leader-follower rig built for under $20,000, gathered demonstrations from which a learned policy performed tasks such as slotting a battery with 80-90% success after about ten minutes of demonstrations.⁶ Companies make the same argument. In 2025 the home-robot company 1X said that planned in-home tests of its Neo Gamma robot would rely on remote operators who can view the robot's cameras in real time and take control of its limbs, and that early adopters would help build a dataset for training its in-house models.¹⁵ Mundane presents its exoskeleton the same way, as a working tool that also records skilled work for later automation.¹³

This resembles Tachi's original division of labour, in which robots worked autonomously under supervisory control and a human switched into telexistence for tasks beyond them.¹ Either way, a person directs the robot from outside, an arrangement distinct from the proposed Human–AI merger.

§Limitations

Latency and network stability limit everything else. Operators facing delay slow down into a "move-and-wait" strategy, first reported in research on delays that began in the 1960s, and methods for stable force reflection over delayed links, based on passivity, appeared from the mid-1980s, initially assuming a constant delay.⁴ iCub3's developers found that lagging video made operators more cautious and raised their cognitive load. They suggested that feeling contact before seeing it could help, while noting that haptic and visual feedback arriving out of step adds load of its own.⁹

Situational awareness is narrower through a robot. Judges reported that the narrow field of view of cameras and headsets limited their perception around the robot. Robots collided with obstacles, and one crashed after backing over a rock, a failure the analysis suggests a rear camera or collision sensors might have prevented.³ Haptics remains the weakest sense: at the XPRIZE, feeling forces and touch remotely drew the lowest operator scores at the semifinals, and the texture task had the lowest success of the finals manipulation tasks.² Touch devices of other kinds are covered under Sensory augmentation.

Cybersickness and fatigue limit who can operate and for how long. XPRIZE operator judges had to declare themselves not prone to virtual-reality sickness.³ The NimbRo team reported that among more than 150 people who used its system, very few complained of motion sickness and some operated it for over 90 minutes with only mild fatigue, but concluded that removing motion sickness for every operator might be impossible.⁵ The evaluations cited here run from minutes to just over 90 minutes; none measures a full working shift.

Complexity and cost are the last constraint. The most successful XPRIZE teams came from well-funded laboratories and companies, and several less-funded teams did not reach physical testing,³ a pattern familiar from Access and inequality in other expensive technologies.

§Risks

A robot that loses its link mid-task is a hazard in its own right. XPRIZE's semifinal scoring asked whether an avatar remained in a safe and stable position when not actively controlled.²

Avatars that work in homes or workplaces put remote staff behind cameras in private spaces. 1X says its customers decide when an employee may view the robot's surroundings, but the reporting notes that collecting data inside homes raises a range of privacy concerns.¹⁵

The labour effects run in two directions. Tachi has argued that telexistence would let workers do physical jobs from anywhere, letting older and disabled people work on equal physical terms and letting foreign workers take jobs without migrating.¹ The second claim is a forecast about global labour markets, not a result, and the first sits inside the wider debate covered under Disability rights and enhancement. The data argument points the other way: demonstrations collected from skilled operators are the training material for automating their routine tasks. Mundane's own story page describes a future in which "one person can direct a whole crew of robots at once".¹⁶ Whether that makes the operator more valuable or simply fewer operators necessary is not settled by any published evidence.

§Outlook

The competition report's authors wrote that commercialisation requires identifying viable use cases, that everyday virtual travel would need simpler and more affordable systems, and that a possible follow-up competition could impose bandwidth limits, latency, rough terrain and harder bimanual tasks.² Those are the conditions a hazardous-work avatar would actually face.

Behnke and colleagues describe avatar telepresence as the closest existing thing to "beaming".² Unlike the teleportation thought experiment, nothing about the operator moves or is copied: the body stays home and only perception and action travel. As a form of Human enhancement, an avatar leaves the operator's own body unmodified, which keeps it apart from the debates over Morphological freedom. Tachi's "telexistence society" remains a forecast. The open question is narrower and empirical: whether a trained worker, working through a robot over a real network for a full shift, can match their own speed and accuracy closely enough to be worth the machine.

§See also

• Powered exoskeletons
• Sensory augmentation
• Neuroprosthetics
• Brain–computer interface
• Human enhancement
• Morphological freedom
• Space medicine
• Myoelectric prosthetics

§References