In many AV designs they use HD Maps to have static data ahead of time. Both to compare to but offload some of the real time processing. However, as you say non-environmental objects have to be detected in real-time.

With regard to any object and it’s movement whether it be that ball or child they handle things with various progressions of detection and classification fidelity. From dynamic sensing and avoidance to deep learning. That being low, medium and high fidelity detection and classification. Using that ball as an example. Low fidelity is where a mining vehicle detecting that ball, not caring what it is and just stopping. Medium fidelity may be a slow speed shuttle determining it is a ball and planning. High fidelity may be determining what kind of ball and planning to that.