Waymo releases extremely misleading AI benchmark results

Michael DeKort
3 min readOct 1, 2022

--

Waymo statement — https://blog.waymo.com/2022/09/benchmarking-av-safety.html

Reference article — Waymo Claims Its Driver AI Is Safer Than a Human and Backs It Up with Test Results

https://jalopnik.com/waymo-claims-its-driver-ai-is-safer-than-a-human-and-ba-1849598194

While the core apples to apples response time improvement is encouraging, the ability to do something and doing it in enough scenarios, especially crash cases to be L4 or driverless, is the key indicator. And that is in no way satisfied here.

· This is based on an extremely minimal set of crashes.

· Regarding the human crash cases they avoided. They lead you to believe all or most humans would have the same crashes.

· There is no data on disengagements that would have been crashes if the human didn’t disengage. Both in the real-world and in simulation

· No mention of all the scenarios learned especially crash and edge cases.

All of this means there is no significant proof of performance here at all. Thus, making this mostly false confidence inducing hype. If not gross negligence. Particularly because Waymo says they are driverless while not only refusing to provide any meaningful proof, but they also used the courts in the state of California to avoid doing so.

Below are a couple articles that explain my POV in more detail.

By not providing any meaningful proof of being driverless, even fighting doing through a lawsuit, Waymo, Cruise and Gatik are misleading the public, putting their lives at risk, and collapsing

· https://imispgh.medium.com/by-not-providing-any-meaningful-proof-of-being-driverless-even-fighting-doing-through-a-lawsuit-f07790d6f96a

Nostradamus? SAE Autonomous Vehicle Magazine declares I am “Prescient”

· https://imispgh.medium.com/nostradamus-sae-autonomous-vehicle-magazine-declares-i-am-prescient-99325d4c0385

Autonomous Vehicle Industry’s Self-Inflicted and Avoidable Collapse — Ongoing Update

· https://imispgh.medium.com/autonomous-vehicle-industrys-self-inflicted-and-avoidable-collapse-ongoing-update-c21d9e65cc35

The Autonomous Vehicle Industry can be Saved by doing the Opposite of what is being done now to create this technology

· https://medium.com/@imispgh/the-autonomous-vehicle-industry-can-be-saved-by-doing-the-opposite-of-what-is-being-done-now-b4e5c6ae9237

Waymo finally discovers gaming modeling and simulation technology is not adequate

- https://imispgh.medium.com/waymo-finally-discovers-gaming-modeling-and-simulation-technology-is-not-adequate-4b1aace9be95

How the failed Iranian hostage rescue in 1980 can save the Autonomous Vehicle industry

My name is Michael DeKort — I am Navy veteran (ASW-C4ISR) and a former system engineer, engineering, and program manager for Lockheed Martin. I worked in aircraft simulation, the software engineering manager for all of NORAD, a software project manager on an Aegis Weapon System baseline, and a C4ISR systems engineer for DoD/DHS and the US State Department (counter-terrorism). I am currently the Senior Advisory Technical Project Manager for FTI to the Army AI Task Force at CMU NREC (National Robotics Engineering Center)

Autonomous Industry Participation — Air and Ground

- Founder SAE On-Road Autonomous Driving Simulation Task Force

- Member SAE ORAD Verification and Validation Task Force

- Member UNECE WP.29 SG2 Virtual Testing

- Stakeholder USDOT VOICES (Virtual Open Innovation Collaborative Environment for Safety)

- Member SAE G-35, Modeling, Simulation, Training for Emerging AV Tech

- Member SAE G-34 / EUROCAE WG-114 Artificial Intelligence in Aviation

- Member Teleoperation Consortium

- Member CIVATAglobal — Civic Air Transport Association

- Stakeholder for UL4600 — Creating AV Safety Guidelines

- Member of the IEEE Artificial Intelligence & Autonomous Systems Policy Committee

SAE Autonomous Vehicle Engineering magazine editor calling me “prescient” regarding my position on Tesla and the overall driverless vehicle industry’s untenable development and testing approach — (Page 2) https://assets.techbriefs.com/EML/2021/digital_editions/ave/AVE-202109.pdf

Presented the IEEE Barus Ethics Award for Post 9/11 DoD/DHS Whistleblowing Efforts

--

--

Michael DeKort
Michael DeKort

Written by Michael DeKort

Non-Tribal Truth Seeker-IEEE Barus Ethics Award/9–11 Whistleblower-Aerospace/DoD Systems Engineer/Member SAE Autonomy and eVTOL development V&V & Simulation