Still a long road to fully autonomous passenger cars

Still a long road to fully autonomous passenger cars

By Scott Simmie

 

We hear a lot about self-driving cars – and that’s understandable.

There are a growing number of Teslas on the roads, with many owners testing the latest beta versions of FSD (Full Self-Driving) software. The latest release allows for automated driving on both highways and in cities – but the driver still must be ready to intervene and take control at all times. Genesis, Hyundai and Kia electric vehicles can actively steer, brake and accelerate on highways while the driver’s hands remain on the wheel. Ford EVs offer something known as BlueCruise, a hands-free mode that can be engaged on specific, approved highways in Canada and the US. Other manufacturers, such as Honda, BMW and Mercedes, are also in the driver-assist game.

So a growing number of manufacturers offer something that’s on the path to autonomy. But are there truly autonomous vehicles intended to transport humans on our roads? If not, how long will it take until there are?

Good question. And it was one of several explored during a panel on autonomy (and associated myths) at the fifth annual CAV Canada conference, which took place in Ottawa December 5. InDro’s own Head of Robotic Solutions (and Tesla owner) Peter King joined other experts in the field on the panel.

 

Autonomous Cars

Levels of autonomy

 

As the panel got underway, there were plenty of acronyms being thrown around. The most common were L2 and L3, standing for Level 2 and Level 3 on a scale of autonomy that ranges from zero to five.

This scale was created by the Society of Automotive Engineers as a reference classification system for motor vehicles. At Level 0, there is no automation whatsoever, and all aspects of driving require human input. Think of your standard car, where you basically have to do everything. Level 0 cars can have some assistive features such as stability control, collision warning and automatic emergency braking. But because none of those features are considered to actually help drive the car, such vehicles remain in Level 0.

Level 5 is a fully autonomous vehicle capable of driving at any time of the day or night and in any conditions, ranging from a sunny day with dry pavement through to a raging blizzard or even a hurricane (when, arguably, no one should be driving anyway). The driver does not need to do anything other than input a destination, and is free to watch a movie or even sleep during the voyage. In fact, a Level 5 vehicle would not need a steering wheel, gas pedal, or other standard manual controls. It would also be capable of responding in an emergency situation completely on its own.

Currently, the vast majority of cars on the road in North America are Level 0. And even the most advanced Tesla would be considered Level 2. There is a Level 3 vehicle on the roads in Japan, but there are currently (at least to the best of our knowledge and research), no Level 3 vehicles in the US or Canada.

As consumer research and data analytics firm J.D. Power describes it:

“It is worth repeating and emphasizing the following: As of May 2021, no vehicles sold in the U.S. market have a Level 3, Level 4, or Level 5 automated driving system. All of them require an alert driver sitting in the driver’s seat, ready to take control at any time. If you believe otherwise, you are mistaken, and it could cost you your life, the life of someone you love, or the life of an innocent bystander.”

To get a better picture of these various levels of autonomy, take a look at this graphic produced by the Society of Automotive Engineers International.

Autonomy

Now we’ve got some context…

 

So let’s hear what the experts have to say.

The consensus, as you might have guessed, is that we’re nowhere near the elusive goal of a Level 5 passenger vehicle.

“Ten years ago, we were all promised we’d be in autonomous vehicles by now,” said panel moderator Michelle Gee, Business Development and Strategy Director with extensive experience in the automotive and aerospace sectors. Gee then asked panelists for their own predictions as to when the Level 4 or 5 vehicles would truly arrive.

“I think we’re still probably about seven-plus years away,” offered Colin Singh Dhillon, CTO with the Automotive Parts Manufacturers’ Association.

“But I’d also like to say, it’s not just about the form of mobility, you have to make sure your infrastructure is also smart as well. So if we’re all in a bit of a rush to get there, then I think we also have to make sure we’re taking infrastructure along with us.”

Autonomous Cars

It’s an important point.

Vehicles on the path to autonomy currently have to operate within an infrastructure originally built for human beings operating Level 0 vehicles. Such vehicles, as they move up progressive levels of autonomy, must be able to scan and interpret signage, traffic lights, understand weather and traction conditions – and much more.

Embedding smart technologies along urban streets and even on highways could help enable functionalities and streamline data processing in future. If a Level 4 or 5 vehicle ‘knew’ there was no traffic coming at an upcoming intersection, there would be no need to stop. In fact, if *all* vehicles were Level 4 or above, smart infrastructure could fully negate the need for traffic lights and road signs entirely.

 

Seven to 10 years?

 

If that’s truly the reality, why is there so much talk about autonomous cars right now?

The answer, it was suggested, is in commonly used – but misleading – language. The term “self-driving” has become commonplace, even when referring solely to the ability of a vehicle to maintain speed and lane position on the highway. Tesla refers to its beta software as “Full Self-Driving.” And when consumers hear that, they think autonomy – even though such vehicles are only Level 2 on the autonomy scale. So some education around langage may be in order, suggested some panelists.

“It’s the visual of the word ‘self-driving’ – which somehow means: ‘Oh, it’s autonomous.’ But it isn’t,” explained Dhillon. “…maybe make automakers change those terms. If that was ‘driver-assisted driving,’ then I don’t think people would sleeping at the wheel whilst they’re on the highway.”

One panelist suggested looking ahead to Level 5 may be impractical – and even unnecessary. Recall that Level 5 means a vehicle capable of operating in all conditions, including weather events like hurricanes, where the vast majority of people would not even attempt to drive.

“It’s not safe for a human to be out in those conditions…I think we should be honing down on the ‘must-haves,’ offered Selika Josaih Talbott, a strategic advisor known for her thoughtful takes on autonomy, EVs and mobility.

“Can it move safely within communities in the most generalised conditions? And I think we’re clearly getting there. I don’t even know that it’s (Level 5) something we need to get to, so I’d rather concentrate on Level 3 and Level 4 at this point.”

 

Autonomous Cars

InDro’s Peter King agrees that Level 5 isn’t coming anytime soon.

I believe the technology will be ready within the next 10 years,” he says. “But I believe it’ll take 30-40 years before we see widespread adoption due to necessary changes required in infrastructure, regulation and consumer buy-in.”

And that’s not all.

“A go-to-market strategy for Level 5 autonomy is a monumental task. It involves significant investments in technology and infrastructure – and needs to be done so in collaboration with regulators while also factoring in safety and trust from consumers with a business model that is attainable for the masses.”

What about robots?

Specifically, what about Uncrewed Ground Vehicles like InDro’s Sentinel inspection robot, designed for monitoring remote facilities like electrical substations and solar farms? Sentinel is currently teleoperated over 4G and 5G networks with a human controlling the robot’s actions and monitoring its data output. 

Yet regular readers will also know we recently announced InDro Autonomy, a forthcoming software package we said will allow Sentinel and other ROS2 (Robot Operating System) machines to carry out autonomous missions.

Were we – perhaps like some automakers – overstating things?

“The six levels of autonomy put together by the SAE are meant to apply to motor vehicles that carry humans,” explains Arron Griffiths, InDro’s lead engineer. In fact, there’s a separate categorization for UGVs.

The American Society for Testing and Materials (ASTM), which creates standards, describes those tiers as follows: “Currently, an A-UGV can be at one of three autonomy levels: automatic, automated, or autonomous. Vehicles operating on the first two levels (automatic and automated) are referred to as automatic guided vehicles (AGVs), while those on the third are called mobile robots.”

“With uncrewed robots like Sentinel, we like to think of autonomy as requiring minimal human intervention over time,” explains Griffiths. “Because Sentinel can auto-dock for wireless recharging in between missions, we believe it could go for weeks – quite likely even longer – without human intervention, regardless of whether that intervention is in-person or virtual,” he says.

“The other thing to consider is that these remote ground robots, in general, don’t have to cope with the myriad of inputs and potential dangers that an autonomous vehicle driving in a city must contend with. Nearly all of our UGV ground deployments are in remote and fenced-in facilities – with no people or other vehicles around.”

So yes, given that InDro’s Sentinel will be able to operate independently – or with minimal human intervention spread over long periods – we are comfortable with saying that machine will soon be autonomous. It will even be smart enough to figure out shortcuts over time that might make its data missions more efficient.

It won’t have the capabilities of that elusive Level 5 – but it will get the job done.

InDro’s take

 

Autonomy isn’t easy. Trying to create a fully autonomous vehicle that can safely transport a human (and drive more safely than a human in all conditions), is a daunting task. We expect Level 5 passenger vehicles will come, but there’s still a long road ahead.

Things are easier when it comes to Uncrewed Ground Vehicles collecting data in remote locations (which is, arguably, where they’re needed most). They don’t have to deal with urban infrastructure, unpredictable drivers, reading and interpreting signage, etc.

That doesn’t mean it’s easy, of course – but it is doable.

And we’re doing it. Stay tuned for the Q1 release of InDro Autonomy.

 

Percepto’s autonomous drone-in-a-box

Percepto’s autonomous drone-in-a-box

By Scott Simmie, InDro Robotics

 

If you’re in the drone industry, you’ve likely heard the phrase: “Drone-in-a-box.” If refers to an autonomous system where a drone nests inside an enclosure for charging and safe harbour – and is regularly dispatched for automated missions. Usually, those missions are pre-programmed and involve inspection, surveillance, or change detection. Manual missions can be carried out when necessary, but the real point is automation.

The idea is that missions are carried out repeatedly, with a human simply monitoring from a remote location. Remotely could mean inside a building on an industrial site where the drone is based. But, because these emerging systems operate using LTE to control the drone and communicate with the software, they can be operated from hundreds or even thousands of kilometres away (providing you have permission to operate Beyond Visual Line of Sight and an available LTE network). With missions carried out automatically and on a regular schedule, this makes for vastly more efficient inspections, surveillance, tracking of construction progress, etc.

It beats requiring a pilot on site, and the drone never suffers from fatigue. The advantages for Enterprise clients are immediately apparent: Inspections, surveillance or general monitoring take place like clockwork, with all relevant data stored for easy access and interpretation. But think also of the edge such systems could provide for First Responders: A drone could be automatically dispatched to the location of a 9-1-1 call or critical incident. Video or thermal imagery can be securely live-streamed to decision makers down the line, regardless of where they’re located. The potential of such systems is unlimited.

At InDro Robotics, we’re no strangers to this concept. In fact, we’ve got a few things under the hood in this regard. But we like to acknowledge and celebrate success in this field. And so today’s post will focus on Percepto – the world’s leader in drone-in-a-box solutions. The company has a proven system, currently deployed in more than 100 locations around the globe. We recently had an opportunity to see a demo of the Percepto system, hosted by Canadian distributors RMUS (Rocky Mountain Unmanned Solutions).

Percepto’s autonomous drone system

The Percepto website outlines its offering with this statement: “Changing the way visual data is collected and analyzed, Percepto AIM is the only end-to-end inspection and monitoring software solution that fully automates visual data workflows, from capture to insights.” AIM stands for Autonomous Inspection and Monitoring, and is the software integral to the overall package.

This Percepto video provides a good overview:

It’s one thing to see a company video, quite something else to see that system in person. We were part of a briefing with Percepto’s Ehud Ollech (Head of Business Development) and Shykeh Gordon (VP Global Sales). They demonstrated the AIM software, the Sparrow drone (which comes with a parachute), and much more.

But they started by explaining that this system is purpose-built for major industrial customers, with clients from the mining, solar, oil & gas/petrochemical and utilities sector. And, they said, don’t think of Percepto as a drone company.

“Basically we are a big data analytics company,” said Shykeh, “offering end-to-end inspection and monitoring solutions.” What kind of solutions? This corporate graphic, supplied by Percepto’s marketing department, helps explain:

Percepto Drone

Percepto’s AIM software

During the demonstration, Shykeh and and Ehud walked us through the AIM (Autonomous site Inspection and Monitoring) software. It’s a browser-based system that allows you to program missions, monitor flights in real-time, watch a live stream from the Sparrow drone’s RGB or thermal camera, and take a deep dive into meaningful data. The User Interface is simple, and Percepto says a mission can be planned in as little as three minutes. In fact, they flew a brief mission from indoors with visitors watching from a conference room and visual observers outside. Every aspect of the mission, including a live video feed, was delivered in real-time. We could monitor what the drone was seeing, which is part of the point. And Ehud had the option, if something caught his eye, of stopping for a closer look.

RGB and thermal data is continuously captured during missions, then uploaded to the AWS cloud when the drone comes home to roost. Significantly, the AIM software is capable of change detection – a major feature for many clients. Once a baseline capture of a designated area has been stored in the Cloud, if a subsequent mission detects any changes, anomalies will be flagged. These could include thermal changes, issues with solar panels, oil leaks, a broken window – the list goes on. (The thermal data is radiometric, meaning it provides the actual temperatures measured.) The AI does not always categorize the type of anomaly, but even when it doesn’t it will quickly point out the relevant images for the operator to take a closer look. Percepto can also be integrated with Smart Fences or Pan-Tilt-Zoom cameras and dispatched automatically if something seem amiss.

Percepto Autonomous Drone

“The heart of our system is our software,” says Ehud.

AIM can also integrate data from ground-based robots, such as Boston Dynamics’ Spot. Even a smartphone photo or other image can be added to the mix, providing it contains geolocation data. The system can produce 3D digital twins, with all photogrammetry stitching done in the Cloud by AIM. (Some solutions for automated data capture rely on third-party software for photogrammetry.)  As part of the demonstration, Ehud defined an area of a pile of earth; a volumetric calculation was instantly performed. And this was all while the group was comfortable indoors. We were probably 50 metres from the actual system, but we could have just as easily have been across the planet, assuming LTE connection at the drone end.

Seeing the Percepto drone in a box in action

Percepto Autonomous Drone

After the first mission was complete, we went outside to watch the system in person. It began with the clamshell housing – which Percepto says can withstand a Category 5 hurricane, closed. Once the mission was initiated, it opened up quickly and the Sparrow took off. The system is operational in winds up to 40 kph, with a takeoff/landing limit of 27 kph. Winds during the demo were 24 kph; the Sparrow was rock steady.

Percepto Autonomous Drone

The system does not have obstacle avoidance, but uses ground-based radar to avoid conflicts with crewed aircraft. Altitude parameters, obviously, are programmed when setting the mission.

“Everyone’s waiting for aerial radar to get cheaper and lighter,” said Shykeh.

More sensors to come

The company already is working on a gas detection sensor (aka OGI camera), and is looking at potential LiDAR sensors as well. Maximum flight time is 40 minutes under optimal conditions, but generally flights are limited to 30 minutes. The next generation battery will offer a 20-30 per cent increase in time and range, and charging time in the station – from zero to full – is about 40 minutes.

Percepto Drone

InDro’s view

This is a refined and mature system, well-suited for major corporations with the budget for this kind of data acquisition and interpretation. It’s particularly suited for remote sites – especially sites that do not have staff on site but require persistent monitoring for safety, security or other reasons. Percepto has some very high-profile clients on its roster, including FPL, Koch, Verizon and Enel.

We’re strong supporters of drone-in-a-box solutions – and are actively exploring systems that might be helpful to First Responders. Kudos to Percepto…and stay tuned.