Cruise Robotaxis Struggled To Detect Children, But Cruise Ran Them Anyway, Report Says

Recognizing children poses a significant challenge for autonomous vehicles due to their smaller size and erratic behavior. One might expect autonomous vehicles (AVs) to exercise extra caution around potential child pedestrians. However, it has come to light that GM’s autonomous robotaxi division, Cruise, allowed its AVs to continue operating despite their limited ability to recognize children compared to adults.

According to internal documents reviewed by The Intercept, Cruise was aware of its AVs’ shortcomings in effectively recognizing children but continued to expand its services. This revelation follows the recent revocation of Cruise’s AV taxi permits by the California Department of Motor Vehicles (DMV) after one of its vehicles struck a woman, with Cruise withholding crucial footage of the incident from regulators.

While Cruise publicly acknowledges that accidents cannot be entirely prevented, the company claims to have the lowest risk tolerance for contact with children and prioritizes their safety. However, internal documents suggest a different story. An undisclosed internal safety assessment indicated that “Cruise AVs may not exercise additional care around children,” highlighting the need for improvement in distinguishing children from adults. Simulated tests conducted by the company also raised concerns, with one assessment indicating that “a fully autonomous vehicle might have struck the child.”

The use of machine learning in AV technology, which is commonplace in both fully autonomous robotaxis and advanced driver aid systems, comes under scrutiny. Phil Koopman, an engineering professor specializing in AV safety, suggests that machine learning is fundamentally flawed for safety technology since it only learns from past incidents. This approach may not anticipate scenarios until they occur, potentially posing safety risks.

The disclosure that Cruise overlooked its AVs’ limitations in recognizing children is disconcerting, particularly considering Cruise’s history of technical issues and past failures that were concealed. The temporary suspension of Cruise’s AV operations by the California DMV provides an opportunity to address these concerns.

If autonomous vehicles are to operate independently, they must demonstrate awareness of their surroundings equivalent to that of a human driver. Currently, this level of awareness appears to be lacking in GM’s Cruise service.