You are using a process to create your AVs that will never get close to creating a a true L4 and will cause a significant amount of casualties for no reason. You are doing this wrong.
It is a myth that public shadow driving is a viable method to create AVs. That process can never come close to creating an AV. You cannot drive the one trillion miles or spend over $300B to do so. You also cannot have any more casualties especially a child or family. The path you are on will create thousands of them as you move into more complex and dangerous scenarios. Both due to handover and because you will have to run thousands of accident scenarios thousands of times. The answer is to use FAA practices and aerospace/DoD level simulation for 99% of it.
Impediments to Creating an Autonomous Vehicle https://www.linkedin.com/pulse/impediments-creating-autonomous-vehicle-michael-dekort/
I live in Pittsburgh and would be glad to provide you all the proof needed.