Synthetic Worlds for Improving Driver Assistance Systems

The automotive industry is evolving at a rapid pace, new technologies and techniques are being introduced in order to make the driving experience more pleasant and safer as compared to a few decades ago. But as with any new technology and methodology, there will always be new challenges to overcome. Advanced Driver Assistance systems has attracted a considerable amount of interest in the research community over the past few decades. This research dives into greater depths of how synthetic world simulations can be used to train the next generation of Advanced Driver Assistance Systems in order to detect and alert the driver of any possible risks and dangers during autonomous driving sessions. As Autonomous driving is still in the process of rolling out, we are far away from the point where Cars can truly be autonomous in any given environment and scenario and there are still quite a fair number of challenges to overcome. A number of semi autonomous cars are already on the road for a number of years. These include likes of Tesla, BMW \& Mercedes. But even more recently some of these cars have been involved in accidents which could have been avoided if a driver had control of the vehicle instead of the autonomous systems. This raises the question why are these cars of the future so prone to accidents and whats the best way to over come this problem. The answer lies in the use of synthetic worlds for designing more efficient ADAS in the least amount of time for the automobile of the future. This thesis explores a number of research areas starting from the development of an open source driver simulator that when compared to the state-of-the art is cheaper and efficient to deploy at almost any location. A typical driver simulator can cost between £10,000 to as much as £500,000. Our approach has brought this cost down to less than £2,000 while providing the same visual fidelity and accuracy of the more expensive simulators in the market. On the hardware side, our simulator consist of only 4 main components namely, CPU case, monitors Steering/pedal and webcams. This allows the simulator to be shipped to any location without the need of any complicated setup. When compared to other state-of-the-art simulators \cite{carla}, the setup and programming time is quite low, if a PRT based setup requires 10 days on state-of-the-art simulators then the same aspect can be programmed on our simulator in as little as 15 minutes as the simulator is designed from the ground up to be able to record accurate PRT. The simulator is then successfully used to record accurate Perception Reaction Times among 40 subjects under different driving conditions. The results highlight the fact that not all secondary tasks result in higher reaction times. Moreover, the overall reaction times for hands were recorded at 3.51 seconds whereas the feet were recorded at 2.47 seconds. The study highlights the importance of mental workloads during autonomous driving which is a vastly important aspect for designing ADAS. The novelty from this study resulted in the generation of a new dataset comprising of 1.44 million images targeted at driver vehicular interactions that can be used by researchers and engineers to develop advanced driver assistance systems. The simulator is then further modified to generate hi fidelity weather simulations which when compared to simulators like CARLA provide more control over how the cloud formations giving the researchers more variables to test during simulations and image generation. The resulting synthetic weather dataset called Weather Drive Dataset is unique and novel in nature as its the largest synthetic weather dataset currently available to researchers comprising of 108,333 images with varying weather conditions. Most of the state-of-the-art datasets only have non automotive based images or is not synthetic at all. The proposed dataset has been evaluated against Berkeley Deep Drive dataset which resulted in 74\% accuracy. This proved that synthetic nature of datasets are valid in training the next generation of vision based weather classifiers for autonomous driving. The studies performed will prove to be vital in progressing the Advanced Driver Assistance systems research forward in a number of different ways. The experiments take into account the necessary state of the art methods to compare and differentiate between the proposed methodologies. Most efficient approaches and best practices are also explained in detail which can provide the necessary support to other researchers to set up similar systems to aid in designing synthetic simulations for other research areas.