Demographic skews in training data create algorithmic errors

Every good model relies on training data that reflect what it seeks to predict. For uses like facial recognition, models need enough cases from each demographic group to learn how to identify members accurately. And when making forecasts, like trying to predict successful hires from recorded job interviews, the proportions of each group in training data should resemble those in the population. Many businesses compile private training data. One study of three programs that identify sex in photos found far more errors for dark-skinned women than for light-skinned men.