By Scott Simmie

We’ve all seen the videos on LinkedIn or elsewhere by now: A humanoid robot, doing incredible gymnastic feats. Or perhaps it’s toiling away in a factory, moving boxes from one location to another – maybe even tightening a bolt. They’re not AI videos – they (or at least most of them) are indeed real.

They leave an indelible impression – that a humanoid can simply be dropped into a complex environment and immediately get to work on its own. That day will undoubtedly come, but it’s at least several years down the road. And so while those videos are real, they’re not actually realistic. The actions these robots are carrying out represent countless hours of coding and simulation work for each and every specific task. Humanoids are definitely not “Plug And Play” – and won’t be for quite some time.

For InDro, that gap between existing capabilities and the demand for humanoids that can do more represented both a challenge, and an opportunity. What if, we thought, we could create a system where a human being could remotely control a humanoid with a highly intuitive interface? What’s more, what if we could use that system to easily train a humanoid to carry out those same tasks in the future completely on its own?

That’s precisely what we did. Using a combination of our InDro Cortex brain-box, our InDro Controller software interface, and a Virtual Reality headset and hand controllers (plus a lot of our own coding), we’ve developed a system that allows a humanoid to be fully operated by a human being who can see what the robot sees in 3D. This not only allows these modified humanoids to be put to use in an environment where you wouldn’t want to send people – but can be used to train the humanoid to carry out repeatable tasks autonomously in future.

We recently demonstrated these capabilities at the big Mobile World Congress in Spain. Ericsson invited us to display as a way to highlight the power that its private 5G networks enable. Below, Head of R&D Sales Luke Corbeth explains the basics.

INDRO’S VR HUMANOID

Trying to control a stock humanoid with a factory remote is usually a challenging and non-intuitive task. It can be cumbersome, frustrating work.

Now picture instead donning a 4K Virtual Reality headset and holding hand controllers. You can see what the robot is seeing. The arms follow where your arms move. The hands clasp when you want them to; the wrist articulates. And if you want even finer control over the hands, there’s another option we’ve programmed.

“We support hand tracking as well, so you could ditch the controllers and have the headset track your hand movements and finger positions themselves,” explains InDro’s Front End Developer, RJ Bundy. In other words, the headset’s camera translates your precise hand and finger movements directly into action on the humanoid. What’s more, you can monitor multiple data streams through the headset using Augmented Reality.

Already, we have clients putting these humanoids into hazardous environments.

“We’re seeing strong interest from places like nuclear sites, where you don’t necessarily want to send in a person. It’s a powerful tool for remote telepresence, allowing experts to step into wherever they need to be without actually having to physically be there.”

What’s more, the actions carried out by the human can be used to train the robot to carry out such tasks autonomously down the road.

“Every teleoperative movement can be recorded as motion data,” says Bundy. “We then use that data to train the robot so it learns context, balance – the nuances of real human motion. And then, over time, those recordings help accelerate our autonomy. It allows the robot to repeat tasks more independently without a human having to control it every time.”

The VR humanoid is an impressive step on the road to full autonomy. And it’s made possible by two other InDro innovations.

INDRO CORTEX, INDRO CONTROLLER

InDro is known for our custom robotic builds for industry, research, defence and more. Nearly every single robot (or drone) that leaves our manufacturing facility is outfitted with both InDro Cortex and InDro Controller. (The rare occasions when they’re not are when researchers want to test or build their own solutions.)

InDro Cortex is a very small, very powerful AI compute unit we integrate with the robot and its various sensors. It comes pre-loaded with InDro Controller software, which is our interface with the robot, allowing you to carry out everything from manually controlled missions through to fully autonomous missions, including Simultaneous Localisation And Mapping (SLAM) in unfamiliar environments. We have four tiers of Controller available, depending on use-cases. InDro Controller also enables ultra low-latency operations over 5G, meaning you can operate a robot or drone remotely anywhere there’s a cellular signal (or local WiFi network).

With powerful AI and Machine Vision capabilities, Cortex is the brain behind our devices, including the VR humanoid. With Controller onboard, you simply log in via an encrypted browser. Data from every sensor onboard is at your fingertips in the intuitive console. Want to program a robot for a regular surveillance or asset monitoring mission? It’s simple, including having the robot stop at particular points of interest to inspect gauges, measure temperatures, etc. Need to add an additional sensor? Cortex provides power distribution, sensor fusion, and can instantly detect and integrate any ROS 2-enabled sensor, saving countless hours of integration hassle for clients.

It’s the incredible power – and synergy – of Cortex and Controller that enable all of our powerful robots. And those two provided the foundation which enabled (with a lot of dedicated engineering) our new VR humanoid. It can be put to use immediately with intuitive manual control, learning the tasks it can one day carry out autonomously.

Below: InDro Cortex and InDro Controller also power our new surveillance and inspection quadruped, Prowler. It’s available as shown for $50k CAD and autonomously returns after missions to wirelessly recharge

INDRO’S TAKE

We received incredible feedback at the Mobile World Congress in Spain for our VR humanoid. Attendees instantly understood the ease of operation – and the gap this product fills as we advance toward completely independent, fully autonomous humanoids that will one day be working alongside humans in multiple settings.

“I’m extremely pleased with our Area X.O engineering team and the work it has accomplished in making this VR humanoid a reality,” says InDro Founder and CEO Philip Reece. “We have already shipped to a client for hazardous environment operations, with other VR humanoids being integrated as we speak. It’s a great example not only of the power of Cortex and Controller – but also our ability to build upon these innovations in new and highly useful ways.”