NHTSA’s Framework for Automated Driving System Safety is a Massive Missed Opportunity
The announcement and link — https://www.nhtsa.gov/press-releases/public-comment-automated-driving-system-safety-principles
I am going to start off by making my first positive comment ever about NHTSA on this space. It is saying SOME of the right words with regard validating safety.
Examples
· NHTSA envisions that a framework approach to safety for ADS developers would use performance-oriented approaches and metrics that would accommodate the design flexibility needed to ensure that manufacturers can pursue safety innovations and novel designs in these new technologies.”
· At this stage, NHTSA believes there are four primary functions of the ADS that should be the focus of the Agency’s attention. First, how the ADS receives information about its environment through sensors (“sensing”). Second, how the ADS detects and categorizes other road users (vehicles, motorcyclists, pedestrians, etc.), infrastructure (traffic signs, signals, etc.), and conditions (weather events, road construction, etc.) (“perception”). Third, how the ADS analyzes the situation, plans the route it will take on the way to its intended destination, and makes decisions on how to respond appropriately to the road users, infrastructure, and conditions detected and categorized (“planning”). Fourth, how the ADS executes the driving functions necessary to carry out that plan (“control”) through interaction with other parts of the vehicle. While other elements of ADS safety are discussed throughout this Notice, these four primary functions serve as the core elements NHTSA is considering.
Now. . .the devil is in the details. And given NHTSA’ deplorable track record here and the fact that it doubles down on that horrible track record, I am not holding out hope there will be any there there.
Here is a statement that forewarns this- “Further, as small-scale deployments start to appear in the coming years, NHTSA will address unreasonable safety risks that may arise using its defect investigation and remediation authority.” The key word here is “unreasonable”. I guaranty this means NHTSA buys into the grossly negligent myth that people literally have to be harmed or die as human Guinea pigs to create this technology. More on that below.
Horrible Track Record
NHTSA Believes Safety Standards Slow Technology Advancements
· This is absolute nonsense. How does a standard saying thou shall not plow into large stationary objects, like police cars (Yes, you Tesla) impede technology? Why do I care what tech is used to do that? Additionally, history has proven two things over and over and over. Unless safety increase the bottom line industries will not volunteer to use or create best practices. They have to be forced to do it. This usually occurs after increasing tragedies, press coverage, hearings and legislation. The other point involves competition. When the government sets objective, testable and fair safety standards hype drops and competition increases because the playing field is now level.
NHTSA Still Buys into the Development and Test Approach Myth
· It is a myth that public shadow and safety driving can create a legitimate autonomous vehicle. And the lives the process takes are necessary and for the greater good. It is impossible to drive the trillion miles or spend $300B to stumble and restumble on all the scenarios necessary to complete the effort. The process also harms people for no reason. The first safety issue is handover. The time to regain proper situational awareness and do the right thing, especially in time critical scenarios. cannot be provided. Another dangerous area is learning accident scenarios. AV makers will have to run thousands of accident scenarios thousands of times to accomplish this. That will cause thousands of injuries and deaths. The solution is to use proper simulation. That brings us to the nest point.
NHTSA’s Lack of Understanding of Proper Simulation
· “As described in AV 3.0, ADS development does not start with public, on-road testing. Rather, much of the very early testing of prototype ADS by developers is conducted in simulation and/or closed-course (i.e., track) testing environments.13 Public road testing of a prototype ADS typically begins after significant engineering and safety analysis are performed by developers to understand safety risks and mitigation strategies are put in place to address those risks. It is important to note that the development process is generally both iterative and cyclical. A developer does not “graduate” from simulation to track test, and then to on-road testing, and then deployment. Instead, developers will generally continue simulation testing throughout the development process to gain additional experience with various scenarios that may be encountered rarely in the real world. Similarly, track testing designed to resemble scenarios that may be encountered rarely or that would be dangerous to attempt on public roads until later stages of readiness will occur throughout the process, even as on-road testing is occurring. Further, experiences gained from on-road testing will often lead to simulation and/or test track replication of situations encountered on public roads to improve the ADS. In other words, the fact that a vehicle is being tested on public roads does not mean that the vehicle or ADS is nearing deployment readiness and, conversely, the fact that a vehicle is still undergoing simulation or track testing does not mean is it not safe to be tested on public roads.”
· NHTSA has it exactly backwards. They should do exactly what they say should not be done. “A developer does not “graduate” from simulation to track test, and then to on-road testing, and then deployment. Instead, developers will generally continue simulation testing throughout the development process to gain additional experience with various scenarios that may be encountered rarely in the real world.”
· The reason why they and the industry get this wrong is because they are using gaming-based simulation that cannot make anything close to a legitimate digital twin, especially regarding active sensor physics and real-time. Since they assume the great looking simulation, they see is the best there is, and knowing it is not good enough to replace most of the real-world, they go right back to relying on that real-world. Even though it is untenable from a time, cost and safety POV. More in my articles below.
In the end all NHTSA is doing is double down on an approach that will hasten the industries current slide into bankruptcy and injuring or harming people needlessly.
More details in my articles here
SAE Autonomous Vehicle Engineering Magazine — Simulation’s Next Generation (featuring Dactle)
The Autonomous Vehicle Industry can be Saved by doing the Opposite of what is being done now
RAND’s “Safe Enough” Driverless Vehicle Industry Report Makes Things Worse
Autonomous Vehicle Industry’s Self-Inflicted and Avoidable Collapse — Ongoing Update
Proposal for Successfully Creating an Autonomous Ground or Air Vehicle
Simulation can create a Complete Digital Twin of the Real World if DoD/Aerospace Technology is used
Simulation Photorealism is almost Irrelevant for Autonomous Vehicle Development and Testing
Autonomous Vehicles Need to Have Accidents to Develop this Technology
Using the Real World is better than Proper Simulation for AV Development — NONSENSE
The Hype of Geofencing for Autonomous Vehicles
SAE Autonomous Vehicle Engineering Magazine — End Public Shadow/Safety Driving
· https://www.nxtbook.com/nxtbooks/sae/ave_201901/index.php
My name is Michael DeKort — I am a former system engineer, engineering and program manager for Lockheed Martin. I worked in aircraft simulation, the software engineering manager for all of NORAD, the Aegis Weapon System, and on C4ISR for DHS.
Key Industry Participation
- Lead — SAE On-Road Autonomous Driving SAE Model and Simulation Task
- Member SAE ORAD Verification and Validation Task Force
- Member DIN/SAE International Alliance for Mobility Testing & Standardization (IAMTS) Sensor Simulation Specs
- Stakeholder for UL4600 — Creating AV Safety Guidelines
- Member of the IEEE Artificial Intelligence & Autonomous Systems Policy Committee (AI&ASPC)
- Presented the IEEE Barus Ethics Award for Post 9/11 Efforts
My company is Dactle
We are building an aerospace/DoD/FAA level D, full L4/5 simulation-based testing and AI system with an end-state scenario matrix to address several of the critical issues in the AV/OEM industry I mentioned in my articles below. This includes replacing 99.9% of public shadow and safety driving. As well as dealing with significant real-time, model fidelity and loading/scaling issues caused by using gaming engines and other architectures. (Issues Unity will confirm. We are now working together. We are also working with UAV companies). If not remedied these issues will lead to false confidence and performance differences between what the Plan believes will happen and what actually happens. If someone would like to see a demo or discuss this further please let me know.
