Urban AI systems lack accountability and risk measures, unlike commercial AI with national oversight. Urban AI is a complex socio-technical system, not just technology, and needs regulation due to its deep integration in public infrastructure, services, and daily life. The United Nations (UN) 2030 Agenda for Sustainable Development Goals (SDGs), specifically SDG 11, aims to create inclusive, safe, resilient, and sustainable cities. Over the past several decades, AI has contributed to the SDGs by improving planning, reducing congestion, and enhancing public services1. However, it also introduces new systemic risks and governance complexities for cities2. Compared to conventional AI systems, urban AI governance is particularly complex because the municipal government often acts as both deployers and regulators, with blurred lines of responsibility. Furthermore, urban AI is embedded in critical public infrastructure such as power grids and transportation systems, where failures could lead to serious societal, political, and economic consequences. Figure 1 outlines key applications, security threats, and policy roles in urban AI systems. It is foreseeable that AI security will become a global priority for sustainable urban development, yet current governance frameworks have not sufficiently addressed these challenges. In this Comment, we conceptualize urban AI security as a socio-technical challenge encompassing two interrelated dimensions: algorithmic accountability and infrastructure security. The former concerns transparency, auditability, and mechanisms for accountability in AI-assisted public decision-making, while the latter involves the robustness and resilience of AI-embedded urban infrastructures against failures and attacks. We first examine the current governance landscape of urban AI and then analyze these two dimensions to identify key risks and policy gaps. Note that this Comment focuses on AI systems deployed in urban governance and infrastructure, rather than general or purely commercial AI applications. The governance vacuum in urban AI Despite the increasing integration of AI into municipal services and infrastructure, there