A deep dive into our Sentinel inspection robot

A deep dive into our Sentinel inspection robot

Oct 21, 2024 | InDro Sentinel, InDro Solutions, Inspection, Scott Simmie, Sentinel

By Scott Simmie

 

Back in 2022 we officially took the wraps off Sentinel – a workhorse robot aimed at remotely inspecting assets like electrical substations.

Before we even designed and built Sentinel, our engineering and sales team put a lot of thought into what would need to go into such a robot. We knew, first and foremost, it had be rugged and capable regardless of terrain. It had to be nearly impervious to inclement weather. It had to be capable of transmitting even dense data from hundreds or thousands of kilometres away from the client. Plus, it had to be equipped with state-of-the-art sensors. Oh yes, it also needed to perform self-diagnostics and send back an easy to read overall health report.

And then our team at Area X.O got to work.

Below: Sentinel

INVENT, ENHANCE, DEPLOY

 

You may have heard that phrase before. It’s a quote from Founder and CEO Philip Reece that succinctly encapsulates what InDro does. It has since become our tagline, and if you see us at a convention you’ll see it on our branding.

The reason we emphasize this now is because InDro remains committed to this ethos. Once we have a product out in the field, we don’t rest on our laurels. We gather learnings from our testing, deployments, and – most importantly of all – our clients to see if we can make an already good product better.

Sentinel featured high-power onboard compute, a 30x optical PTZ camera (complete with wiper!), a thermal sensor and more. It also carried the ROS1 software library onboard which – among other things – allowed Sentinel to instantly recognise any new sensors that were added or replaced. It came equipped with a high-speed modem for remote teleoperations. And it had a pretty good version of our own InDro Autonomy stack.

During a demonstration from Washington State we connected over a 5G network to Area X.O in Ottawa. The two locations were more than 4100 kilometres apart. At the Bellevue end, we had an Xbox controller connected to a laptop. That laptop was logged into a third-party console for remote operations.

At the other end, Sentinel was awaiting instructions. We showed several people the basics of how to operate (and it doesn’t get more intuitive than an Xbox controller) and let them take a spin. You might well think, given the distance involved, there would have been a perceptible lag. There wasn’t (I was there). The response was so instantaneous it was almost like playing a video game. Plus, realtime data – including a video feed – was coming in from Sentinel’s sensors.

 

THE EPRI DEPLOYMENT

 

Testing robots – particularly robots like Sentinel – in a real-world environment is crucial. And, since Sentinel was designed for the remote inspection of critical assets like electrical substations, we caught the attention of EPRI – the Electric Power Research Institute. The non-profit organization is constantly doing research, collaborating with more than 450 private companies across 45 countries globally.

EPRI has multiple research facilities, including one in Lenox, Massachusetts. This particular location features an electrical substation that can be energised, de-energised – and can even simulate rain for testing purposes. So we took Sentinel down there and put it through its paces. We ran it through every variation of testing they could throw at it, including nighttime missions (with the substation both energised and de-energised) as well as with and without rain in each of those scenarios. (Yes, we did daytime missions as well.)

The purpose was to see how well Sentinel could perform in that very much real-world setting. It did very well – though we aso learned some lessons. We discovered that gravel could occasionally get caught between the gears and the treads. And, more importantly, we found that capturing and delivering all that data while doing remote ops put a crushing demand on the compute (a Jetson Xaiver NX).

“We were maxing out the CPU,” recalls Head of R&D Sales Luke Corbeth, who was on that mission. “We were at pretty much 99 per cent usage.”

Over the course of subsequent deployments and testing, we began to explore ways to improve Sentinel.

Below: Data flows through the third-party console we were using at the time to monitor missions

UPGRADED INTERIOR, EXTERIOR

We’ve always had a consistent vision for Sentinel and its use-cases.

“The concept has basically gone unchanged since our initial vision,” says Corbeth. “We envisioned certain industries with an emphasis on the value of preventative maintenance. That has remained the focus.”

But as we encountered new environments and challenges, we worked on ways to improve the build. We are now taking advantage of the latest in AI processing, giving these robots the NVIDIA Jetson AGX Orin Developer Kit for compute. It’s capable of up to 275 trillion operations per second (TOPS) – and is a huge leap forward in compute and AI recognition. Plus, you can throw literally everything at it and it doesn’t break a sweat.

“We’re now able to operate – with all data streaming – at about 30 to 40 per cent of the CPU’s capacity,” says Corbeth.

An optional platform upgrade brings us to the Bunker Pro, which has better weather protection (including skirts that prevent gravel from getting stuck) and longer run time. We moved to the latest models of PTZ and thermal cameras to ensure exceptional resolution and the best imagery in low-light situations. We integrated more pinhole cameras for 360° situational awareness and added a second depth perception camera. We can also add a range finder and two hemispherical LiDAR units, pending client needs. (There’s also the option of safety LiDAR units if Sentinel is deployed around people.)

And that’s just for starters. In addition, InDro’s engineering team can add:

  • New connectivity module for greater bandwidth and lower latency
  • Improved GPS and IMU units
  • The latest InDro Autonomy stack

All of these capabilities ensure Sentinel can operate with precision in unfamiliar and GPS-denied environments. It returns to base and recharges wirelessly without any mechanical connection (thus avoiding a potential point of failure).  It can capture flawless digital twins of entire facilities.

And InDro is in control of all facets of production.

“With the introduction of InDro Forge, we’ve been able to do all of the design and manufacturing in-house,” says Corbeth.

Below: Sentinel combines brains with brawn

InDro Sentinel

MISSION PLANNING

 

Remember how we mentioned originally operating Sentinel via a third-party console? Well, despite getting pretty low latency we found some things were lacking. We wanted a more intuitive and powerful system for controlling Sentinel and setting up autonomous missions. We also wanted a console with bulletproof security.

So we built one. InDro Controller both improves the user experience and adds more power and customization features for the client. We asked Luke to walk us through how easy it is for a client to plan a mission.

“So the robot will always begin its mission starting in the doghouse where it’s actively wirelessly charging, ready to deploy at a moment’s notice with full battery. Deployments can be automated and autonomous, or manually operated,” he explains.

Most clients prefer to have autonomous missions. And InDro is happy to help set up optimal mission planning, with specific repeatable actions at various waypoints. The exceptional AI and machine vision onboard means that Sentinel can not only recognise a gauge – it can understand the reading.

“So we help create the most efficient and safest route for the robot to follow. And at each of its specific waypoints, it’s capturing high-resolution photos, video or thermal data. It will remember, for example, to not only stop but to adjust the camera and zoom in on a specific predetermined asset. You’ll get actionable data with every mission,” adds Corbeth.

Watch both Sentinel and InDro Controller at work in the video below

INDRO’S TAKE

 

There are even more enhancements on the way for Sentinel that we didn’t explore in this post. If you’re curious, you can check out even more features here.

“Sentinel is an exceptionally well-engineered product – where every design decision was made with the client in mind,” says InDro Robotics Founder and CEO Philip Reece. “It has already drawn attention from those seeking a cutting-edge solution to autonomous remote inspections. A US client just purchased several for precisely that use-case, and we’ll be completing those builds shortly.”

Interested in a demo? Get in touch with Luke Corbeth and he’ll set you up.

A deep dive into our Sentinel inspection robot

InDro Robotics launches powerful new Sentinel UGV

Aug 14, 2024 | InDro Autonomy, InDro News, Scott Simmie, Sentinel

By Scott Simmie

 

InDro has taken the wraps off its latest innovation – the all-new Sentinel Uncrewed Ground Vehicle.

Sentinel has been built for inspection, remote teleoperation and surveillance over 5G and 4G networks. Its robust build has been designed to take demanding weather and ground conditions in stride, while capturing virtually any kind of data across a broad range of use-cases.

“That’s what really excites me about Sentinel,” says Head of R&D Sales Luke Corbeth. “It can be used across multiple verticals, ranging from remotely inspecting assets like electrical sub-stations, through to agriculture, mining and more. It’s a workhorse.”

We could list all the sensors and processors onboard (and we will), but it’s the synergy here that counts.

“Sentinel has been very purposely designed and integrated so that virtually anyone can carry out highly complex missions with ease,” explains InDro Vice President Peter King. “And with our InDro Autonomy stack onboard, Sentinel can carry out tasks completely on its own for up to four hours before returning to wirelessly recharge at its base.”

That workhorse is the result of a lot of R&D by our Area X.O engineering team – and also the logical step forward from our original Sentinel V1. That was our original design for remote teleoperated inspection.

But, as you likely know by now, at InDro the innovation never stops.

Below: The new InDro Sentinel, carrying out an autonomous mission

InDro Sentinel

INDRO COMMANDER: SMARt AND VERsAtile

 

What truly sets Sentinel apart from the competition is the synergy of two innovations we’re quite proud of: InDro Commander and InDro Controller. And while Sentinel takes full advantage of these products, they are both robot-agnostic.

InDro Commander is a module that houses Sentinel’s brain – and allows for the seamless integration and power management of other sensors. With multiple USB slots and ROS1 and ROS2 libraries onboard, it makes modifying Sentinel a pleasure rather than a pain. You can, quite simply, plug in a new sensor and it will be detected; no complex coding or wiring required.

“Having Commander on board allows clients to easily modify their robots with different sensors for different applications. This is particularly appealing to those in the R&D community,” says Engineering Lead Arron Griffiths.

Commander also connects with 5G at near-zero latency for remote teleoperations or monitoring dense data during autonomous missions.

“Whether it’s 4G, 5G or WiFi, it does it all,” he adds.

With two state-of-the-art EDGE processors onboard, Commander also has the compute power and AI to take on anything. For autonomy, we’ve integrated the NVIDIA Jetson AGX Orin – a powerful 64GB developer kit. It’s capable of AI calculations at the astounding speed of 275 trillion operations per second (TOPS). Our InDro Autonomy Stack is embedded in Commander for precision autonomous missions. We’ve also outfitted Sentinel with a Realtime Kinematic (RTK) sensor for centimetre-level positioning accuracy.

The other EDGE processor is the ZED Box, which contains an NVIDIA Orin NX (100 TOPS). This carries out multiple tasks, but is specially built to work with depth-perception ZED cameras. There are two of these cameras on board Sentinel, used for AI obstacle detection and avoidance.

Below: A LiDAR-equipped InDro Commander module

Teleoperated Robots

INDRO CONTROLLER AT THE HELM

 

InDro Controller is our custom-built dashboard, and it’s been designed to match perfectly with InDro Commander for an exceedingly powerful but user-friendly means of programming and monitoring missions, robot health, sensor data and more. Operators can easily customize to get precisely the layout they want.

“The dashboard, the cameras, the heads-up display on the autonomous missions – those all can be customised,” says Front End Developer RJ Bundy. “We’re also adding other personal user customisations, like a light and dark mode, metric conversion, schedule missions – all the kinds of features you could want.”

The software resides in Commander and is accessed remotely via an encrypted login system. No one will ever see your missions and data except you – and those you grant the same secure access. (Yes, multiple users can simultaneously monitor a remote operation from locations across the country or across the globe.)

“We’ve created an interface that makes it very hard for a third party to intercept any of those commands or the data coming from the robot to you,” says Bundy.

And while it’s a perfect fit with Sentinel, InDro Controller can be used with any robot.

“It’s an all-in-one data visualization, robot management and robot control software,” he adds. “Whether you’re a student first learning how to use a robot or you’re a commercial giant, you’d be able to manage and maintain all of your robots.”

He’s not kidding. You can intuitively control Sentinel – or any other robot – using an Xbox controller. We’ve done demonstrations more than 4,000 km away from Area X.O where we’ve simply put a controller in someone’s hands. With imperceptible lag, they’re able to operate the robot. (Plus, you can customise the buttons for specific sensor features like pan, tilt, zoom, etc.)

Autonomous missions – including complex actions like stopping at a precise location, tilting a camera to a gauge and then zooming in and capturing the readout – are easily plotted using Controller. Once your mission has been set with multiple waypoints and actions, it’s a single click to repeat it. (And, of course, you can save as many different missions as required.)

The InDro Autonomy stacks are pre-loaded, meaning location doesn’t really matter.

“We have a GPS-based autonomy – which is better for outdoors – and then we have a SLAM- (Simultaneous Localisation and Mapping) based autonomy which is ideal for indoors,” says Head of R&D Sales Corbeth.

There are actually two versions of InDro Controller available; a simpler version is still full-featured but does not ship with InDro Autonomy, meaning it’s ideal for those in the R&D space who want to test their own autonomy software. Several academic institutions have already been using this version, and feedback has been excellent.

And the best news? Both versions will only become even more sophisticated over time.

“We’re heavily invested in continuously improving the software,” says Corbeth. “So regardless of which version you’re shipped, know that this is something that InDro Robotics is constantly developing and improving with client feedback in mind to ultimately provide the best mission planning, teleoperations and development software tool in the robotics industry.”

Below: A screenshot from InDro Controller, showing the path of an autonomous mission; the second image shows realtime data flowing through its customizable dashboard

Robot Teleoperation
Teleoperations with Robots

LONG DURATION, REPEATABLE MISSIONS

It’s already clear Sentinel has the brains. But it also has the brawn to match. Whether the task is inspection, surveillance or other forms of data acquisition, Sentinel has been designed from the ground up to consistently deliver. Featuring a 1500-watt dual drive system and tracks for locomotion, it’s ready for any terrain – from industrial environments through to farm fields and even snow and ice.

With a 120kg load capacity and 30° climbing angle, Sentinel can power its way over pretty much anything. We’ve even designed and manufactured custom metal guards to prevent stones from getting their way between the drive gears and the treads – a common issue with tracked vehicles.

That strength is supplemented by a comprehensive array of sensors for obstacle detection, Simultaneous Localisation and Mapping (SLAM), high-resolution photos and video – and more.

 

ZOOM, ZOOM, ZOOM

For starters, there’s a 30x optical Point-Tilt-Zoom camera that can be manually operated via the InDro Controller UI. Every time the pilot stops and captures data, all of that information – robot position, camera angle, zoom settings etc. are stored as a Point of Interest. All of those POIs from that initial manual mission are saved, meaning the next time Sentinel is deployed the operator can simply recall the mission and launch Sentinel for an autonomous run.

In fact, we recently did a demo of Sentinel for a government official. We had previously captured photos of five different objects on a path at Area X.O and had saved that mission. During the demo, the robot deployed and autonomously stopped at all five locations, replicating the original shots. This is the kind of feature that’s exceedingly helpful for spotting anomalies, change detection, etc. Of course, the built-in AI can also identify people, cars, trucks, etc. (In fact, we could set Sentinel to trigger an alert if a human is spotted somewhere they’re not supposed to be, a useful feature for both safety and security applications.)

For the operator, whether they’re manually piloting or simply monitoring an autonomous mission, four pinhole cameras provide a 360° view of surroundings for situational awareness. Two ZED 3D depth cameras (one at the front, one at the rear) are on the constant lookout for obstacles. As an additional layer of safety, engineers added two 2D safety-rated LiDAR sensors specifically for detecting people or other objects that might unexpectedly move into Sentinel’s path. If that happens, the robot will automatically stop, then seek an alternate path around the person or object. (It can also be set to simply go to a dead stop.)

Sentinel is also outfitted with two powerful Robosense 3D LiDAR units, enabling it to operate in GPS-denied environments, capture precision scans – and basically understand even a completely unfamiliar environment. Robosense has become our LiDAR of choice, for the reasons outlined in this recent post.

Below: InDro recently built and shipped this fleet for swarm robotics research; all are equipped with Robosense LiDAR units

Robosense LiDAR Swarm

WIRELESS SELF-CHARGING

 

Using Sentinel for remote ops? Many of our clients will be. And there’s not much point in deploying a robot at a remote location if someone has to go and plug it in to charge or run diagnostics.

Sentinel returns autonomously to its home base, where a wireless charger awaits. The robot understands where it needs to position itself for optimal charging, snugging up close to (but not touching) the induction charging system. Many remotely operated robots rely on making a mechanical connection with charging equipment, which introduces wear-and-tear and additional points of failure.

“Mechanical charging systems fail a lot, and we wanted to avoid that,” explains InDro VP Peter King.

InDro Controller also carries out diagnostics remotely, reporting back on parameters such as latency, battery strength, sensor connection and other indicators of overall robot health. The operator can literally be hundreds (or even thousands) of kilometres from the robot and see its condition with a quick glance at the UI.

“Sentinel can be customised for any client directly by us,” says Engineering Lead Arron Griffiths. “But having Commander on board allows clients to easily modify their robots with different sensors for different applications. This is particularly appealing to those in the R&D community.”

Below: On the shoulders of giants – the original Sentinel V 1.0. InDro has incorporated new AI compute power, sensors and other learnings to take a massive leap forward

Sentinel Inspection Robot

INDRO’S TAKE

 

The new Sentinel is more powerful, and more suitable to broader use-cases, than our first version of this robot. Don’t get us wrong; the original Sentinel was (and still is) a great machine. But this version is equipped with more powerful EDGE processing, and AI with greater capabilities. It’s also laden with additional and newer sensors, RTK positioning, and a more powerful base platform.

“The new Sentinel is a perfect fusion of industry-leading hardware and software – including InDro Commander, and the new InDro Controller and InDro Autonomy stacks,” says Indro Robotics Founder and CEO Philip Reece. “Our engineering team has put a lot of thought into making this robot capable of highly complex missions – yet easy to use and modify. Sentinel will be a perfect fit for multiple verticals and use-cases, and we look forward to our first deployments.”

Want to learn more about Sentinel – or even arrange to take it for a remote test drive? You can contact Head of R&D Sales Luke Corbeth here.