According to Sky News:

Self-driving cars are generally safer than cars driven by humans, according to a new study released on Tuesday. However, the cars were found to be more accident-prone in certain scenarios.

• at dawn
• at dusk
• during the day and at night
• when turning and going straight
• when starting and stopping
• when they don't have an internet connection
• when they don't have a driver

Otherwise self-driving cars are prefectly safe.