We have witnessed remarkable technology advancements and competitions in autonomous and connected vehicles. There has been a vigorous technological development effort in the past few years to introduce self-driving smart vehicles for a connected world. If applied correctly, these technologies can enable solutions to help city transportation systems improve the economics of transportation, environmental concerns, and quality of life for everyone. The problem requires a holistic approach. While technological development has been intriguing, competitive, and vigorous, the industry’s advances on the issues of safety, risk, and reliability have been dismal. Despite its limited uses and manufacturers’ stated goal of making autonomous cars demonstrably safer than an average human-controlled car, several accidents and near-misses have already occurred. The mean distance driven to an unsafe condition, near miss or accident has been far shorter than the conventional road vehicles. While the public at large is intrigued about these technologies, the safety concerns are profound. This article is aimed to review the safety of AV systems from design and manufacturing, society and ethics, advancement of the safety/reliability technologies assessing their readiness and a review from academic point of view to understand the area for further research.