The hard part on self driving is all the context cues that a human would get and that are hard for machines to encode. Like, for example, slow down when you see a school bus because of children nearby.

However once you get there a machine would be fully aware 100% of the time, which is simply not true for humans.