It's just going to be as good as asking a human to explain its behaviour. It might be a semi-accurate interpretation of its actions based on its internal knowledge, but it's never going to be the actual thing. The actual decision making inside a neural network is fundamentally not something you could simplify into language exactly.
Yeah, it's not clear if we will be satisfied with those explanations: "I decided to slow down because I see these (seemingly random and irrelevant) objects around me", because during training on trillions of video frames the model has learned it should slow down when similar object configurations are present to reduce the chance of an accident by 0.0002%.
Even if the given reason is simple, like "I decided to slow down because the car in front of me is red", explicit override of the learned rule ("don't slow down when you see red cars") might potentially increase the chance of the accident by a lot more than 0.0002% because we are messing with the model decision making process.
