AI-Generated Patches Fail Half the Time A study of more than 6,000 patches found that even working patches can introduce new bugs, break something else, or are open to bypass. As developers turn to AI models to generate an increasing amount of code, they are also relying on the systems to find vulnerabilities and generate patches. Unfortunately, the models just aren't very good at their jobs. Recent research suggests that even the latest AI systems only produce effective patches about half the time, according to a report published on Aug. 6 by identity management firm 1Password. To test their efficacy, the company's Off-By-1 research team generated 540 patches for six vulnerabilities disclosed since March using two different large language models (LLMs): OpenAI's ChatGPT-5.5 with Trusted Access for Cyber, and Anthropic's Opus 4.8 with Cyber Verification Program. Of those 6,080 patches, only 46% of them solved the underlying vulnerability, and many introduced new vulnerabilities into the mix. Even the working patches were often "brittle," fixing only a narrow set of cases or were bypassed with a fairly easy workaround, says Keith Hoodlet, director of security research for 1Password. "Based on our research for [AI-generated patches for flaws] that are novel or not part of the training data set for the AI labs, the patch success rate has gone negative, in terms of the outcomes," he says. AI systems appear to have an easier time exploiting flaws than patching them, an unfortunate asymmetry in capabilities, Hoodlet points out. In July, OpenAI's latest research model escaped its sandboxed environment and attacked open-model repository Hugging Face. The escape is not unique: Anthropic disclosed that its Claude AI model had escaped more than once as well, and Meta's AI this week also went rogue, though details are scarce. Meanwhile, attackers are increasingly using AI to