Forecasting and Preventing Human Errors

Human errors remain the main source of incidents. They can lead to fatalities, traffic accidents, or product defects and cause high economic and social cost. While some errors can still be corrected if they are detected in time, many human errors cause high costs as soon as they occur or are even irreversible. In these cases, it is very important to recognize human errors before they occur. The goal of this project is therefore to develop methods based on artificial intelligence that forecast human errors from video data. We focus on erroneous and unintentional human actions and we aim to support humans to avoid them. In order to achieve this goal, we aim to solve three tasks jointly. We aim to develop methods that forecast human motion and intention with a very low latency such that unintentional actions can be recognized before they occur. Without the capability to interfere, however, even the best forecasting model does not prevent human errors. We therefore aim to develop a model that generates an auditory feedback if an error is forecast. The feedback, however, should not only warn humans, but also guide them such that they can successfully complete their intended action. Finally, we aim to model how humans will react to the feedback. We thus aim to develop a model that forecasts the motion of humans and objects they interact with, that recognizes human errors before they occur, and that guides the human motion via auditory feedback in order to prevent errors. The model should automatically decide if and what auditory feedback is generated by reasoning how the feedback will affect the motion of persons that are close-by. While we aim to showcase that the developed technology is able to prevent errors before they occur, this technology has the potential to drastically reduce the social and economic costs caused by human errors in the long term.