I would like to address these statements
“Third, we know that this transition is not achievable without testing on public roads. We are committed to anticipating and managing risks that may come with this type of testing, but we cannot — as no self-driving developer can — anticipate and eliminate every one.”
“In the months since, we have undertaken a top-to-bottom review of ATG’s safety approaches, system development, and culture. We continue to support the National Transportation Safety Board’s investigation into the Tempe crash. We have taken a measured, phased approach to returning to on-road testing, starting first with manual driving in Pittsburgh.”
I am afraid you need to be far more principled.
It is a myth that public shadow driving is the best or only solution to create a fully autonomous vehicle. The fact is it will be impossible to get anywhere close to a driverless state using this method. First there is the time and money that would be required. It is not possible to drive and redrive, stumble and restumble on all the scenarios necessary to complete the effort. That effort would require one trillion miles to be driven and over $300B to be spent by each AV maker. The other problems involve safety. Those being the running of actual accident scenario to train the AI and L3/handover.
Regarding the training of accident scenarios. That the process will cause thousands of accidents, injuries and casualties when efforts to train and test the AI move from benign scenarios to complex and dangerous scenarios. Thousands of accident scenarios will have to be run thousands of times each. That will cause thousands of needless casualties. Many will eventually be children and families. The other issue is the use of handover. Whether it be in a system under development using public shadow driving or an L3 vehicle in use by the public. In critical scenarios, which includes most accident scenarios, it is impossible no matter what monitoring and notification system is used, to provide the driver with enough time to regain enough situational awareness to do the right thing the right way. This will soon lead to the deaths of not just many more people but children and families. When the public, governments, the press and attorneys’ figure this out they will lose their trust in the industry, question its competence and may impose far more regulation and delay than if the industry self-policed. (The legal expenses that will be incurred as well as the reputational costs will contribute to the bankruptcies I mentioned above.)
The ramifications of these issues is the creation of the exact problem you are trying to avoid. You will never save nearly the lives you hope to, because you will never get close to having a true autonomous vehicle. And you will take lives needlessly in the near eternal fruitless path.
There is a solution though. It is to replace 99% of that public shadow driving with proper simulation and systems engineering and an end-state scenario matrix. To employ a progression from simulation to test track use and finally justified use of the real-world. (It should be noted here that the simulation would clearly have to be informed and validated by the real-world and test track use. Data gathered by hands on driving is encouraged. While hands off public shadow driving validation should be vastly minimized there are scenarios where I would have to be utilized. When that is it needs to be a structured event.)
Proper Simulation — The systems currently in use in this industry are inadequate. They do not have proper real-time architectures and the models being used for vehicles, tires and roads, are not precise enough, especially in degraded conditions. The other issue is the use of driver-in-the -loop simulators without full motion systems. Overall these issues will cause is a significant level of false confidence. You will think the AI has learn properly when it has not. This will not be discovered until analogous real-world scenarios are experienced and expose critical timing and execution gaps. These will lead to accidents or not minimize them as much as they would be otherwise. Keep in mind that the benign scenarios being run now will not encounter these problems. It is not until you run complex or time critical scenarios that the performance envelop of the vehicle, tire or road models are reached and the problems start. Regarding the use of a motion system for manned simulators. This device would be used for simulator shadow driving. Since simulation allows for a 99% increase in efficiency in learning scenarios you can now use professional drivers. The key to the motion system part of this device involves motion queues. Without a motion system the driver will miss critical motion cues or not experience the loss of them when they are expected. This will lead the driver not driving as they would in the real-world. That will lead to the system learning improperly. Examples of when motion cues are important would be when the vehicle is bumped, and no other sensor detects that condition. Or when the vehicles lose traction. An example of the importance of not having expected cues would be when a front wheel drive loses traction. The driver knows that happens because they expect to feel forward motion. Finally, the motion system permits evaluation of motion sickness and passengers feeling of comfort and trust.
How are the simulation issues resolved in addition to having a proper motion system? The leveraging of aerospace/DoD/FAA simulation technology, practices and test criteria, especially as they relate to DoD urban war games, which are directly analogous to complex driving scenarios, as well as how to employ proper model and real-time fidelity.
Proper System Engineering — Utilization of Agile, or a bottom up engineering approach, is an extremely inefficient if not debilitating process when it is employed in complex systems. The key reason for this being wasted time, through not developing in parallel, as well as not attacking the most difficult scenarios up front. (All of which require simulation to execute, including the immediate and endless repletion of these scenarios.) If this approach is not taken the time lost will be extreme if not fatal. Beyond that though is not finding out what design flaws are hidden until the most complex scenarios are experiences. Odds are you will discover flawed design assumptions or executions when you are exposed to the most difficult scenarios. That will usually drive a design and execution change that will need to be employed for many benign scenarios. That could drive significant rework. Even changing your entire design approach or force the relearning of most of your scenarios. (This is where I want to mention the popular and very flawed use of the terms “edge” and “corner” cases. Virtually everyone used them to describe accident scenarios. Accident scenarios are like any other scenario, they just have an outcome we don’t desire. A true edge or corner case is a scenario that should not happen in any possible scenario. Like when a search for the picture of a cat returns a garbage can. The significance of this is the concern that engineers will not search out all the possible accident scenarios because they are deemed on the edges or corners of the core set. Thus, giving people an excuse to do all the required due diligence.)
End-state Scenario Matrix — Beyond providing the scenario data to effect the systems engineering approach I mentioned above is the need for everyone to know what done looks like as early as possible. Form the validators like governments and insurance companies to those trying to produce these systems. Given the scenario set would be massive, with millions of variations (to include ensuring AI does not have perception errors) the effort to do this needs to be just as significant and provide the most efficient core scenario sets to verify every progressive state of completion. From legitimate geofencing to legitimate L4 and L5. This test set would need to be informed from a multitude of data source and domains. It would need to reflect the highest level of due diligence the world community can muster. And ensure that the requisite levels of safety are attained and proven. Finally, it would have to be mapped to and synced with the simulation/simulator system I mentioned above.