Estimating Driver Drowsiness & Distraction with Deep Learning & Computer Vision

There were nearly 37,500 traffic accident fatalities in 2016 in the US., with distraction and fatigue responsible for 10% and 3% of avoidable fatalities. Two seconds of driver distraction is all it takes to double the probability of a collision.

Several car companies are attempting to tackle this issue, but their methods are proprietary and not available for the benefit of most drivers. The cheapest and most reliable way of detecting driver drowsiness or distraction is with a camera and image processing. Other solutions are either too expensive or unrealistic (think EEG electrodes or unreliable biosensors).

In the real world, a python program wouldn’t be very useful because drivers don’t drive with their laptops wide open, so we decided to create an iPhone App that would serve as a prototype for the real-world application of the trained model using the OpenCV framework, a precompiled Dlib library for the image processing operations and CoreML tools for converting the Keras model to a format deployable to iOS using CoreML.

Specifically, a colleague named Shail Shah and I:

• Trained a deep learning model to detect driver facial annotations in a small Spark cluster using Keras/Elephas with a dataset of 200,000 images

• Produced an algorithm to interpret facial annotations to estimate driver drowsiness or distraction with OpenCV. Based this off a subjective measure devised by Swedish psychologists called the Karolinska Sleepiness Scale going from “Extremely Alert” (1) to “Extremely Sleepy, can’t keep awake” (10)

• Developed an iPhone app to be used on a car’s dash and tested the algorithm on a driver video dataset.


With the growing increase of traffic accidents due to driver drowsiness and distraction, it is crucial to develop methods for vehicles to ensure our safety by understanding our mental state, such as by analyzing our facial expressions. To address this problem, I created an algorithm that accurately predicts driver drowsiness and distraction that could run on an iPhone.