Computer engineers at the University of Wisconsin-Madison have developed a new automated dataset that can help pinpoint why facial recognition fails, helping developers improve the accuracy and reliability of these systems. Facial recognition systems are already a huge part of daily life and are becoming even more common. They safeguard data on our phones, speed us through airport security, and help locate missing persons, among other uses. Despite their prevalence, these systems are still not fail-proof. Change your hair color, grow a goatee, get a sunburn or wear a little extra makeup and these digital gatekeepers might struggle to recognize you. Facial recognition developers are not unaware of these problems; what they struggle with is how to pinpoint, understand and fix the types of changing attributes that cause the systems to fail. That’s why engineers led by Guruprasad Viswanathan Ramesh, a PhD student in electrical and computer engineering, have developed a new automated dataset called CounterFace that can be used to evaluate the performance of facial recognition systems. The UW-Madison team, which also included Kassem Fawaz, an associate professor of electrical and computer engineering, and Ramya Korlakai Vinayak, an assistant professor of electrical and computer engineering, recently presented the project at the ACM Conference on Fairness, Accountability, and Transparency held in Montreal in late June 2026. To evaluate facial recognition systems, developers usually test them on large datasets of human faces from different identities, but these datasets can’t be used to pinpoint identification failures. A less commonly used alternative is a dataset containing pairs of human faces in which one attribute is changed—for instance, images showing someone with and without glasses. Such datasets allow developers to reason why their systems fail. Gathering or producing the huge sample of paired images needed to test the systems, however, is time- and cost-prohibitive.