Autonomous Vehicle News
In the European Union’s proposed Artificial Intelligence Act (AI Act) the concept of human oversight is one of the main mechanisms intended to prevent harmful AI outcomes. The promise of human oversight is for humans to notice and correct for any kind of wrong algorithmic advice. Other studies have shown that human oversight has failed to prevent discriminatory decisions in policing, medical applications and felony sentencing. First, the AI Act should acknowledge that it may not always be possible to make human oversight work successfully. Secondly, within the existing conformity assessments for high-risk AI applications, the AI Act should mandate randomised controlled trials to determine if and how human oversight can be successful.