How Cities Use Algorithmic Impact Assessments for AI

This article explores how municipal governments utilize Algorithmic Impact Assessments (AIAs) to evaluate artificial intelligence tools before deploying them in critical public sectors. As cities increasingly adopt automated systems to manage resources, AIAs serve as essential accountability frameworks to prevent bias, ensure transparency, and protect civil rights. Below is a straightforward examination of the AIA process and its specific applications in public housing, education, and policing.

Understanding the AIA Process

An Algorithmic Impact Assessment is a formal evaluation process conducted by a government agency before acquiring or building an automated decision-making system. Inspired by environmental impact statements, AIAs force municipalities to pause and analyze the potential societal consequences of an algorithm.

The standard municipal AIA process involves four main steps:

  1. System Definition: The agency defines the problem the AI is meant to solve and outlines how the algorithm functions.
  2. Data Evaluation: Officials audit the training data for historical biases, inaccuracies, or demographic imbalances.
  3. Public Consultation: Cities release the assessment to the public, allowing community members, advocacy groups, and independent experts to raise concerns.
  4. Mitigation Planning: The agency establishes safeguards, human-in-the-loop oversight, and ongoing auditing schedules to correct errors that arise after deployment.

Evaluating AI in Public Housing

Municipal housing authorities increasingly use algorithms to allocate limited public housing units, screen tenants, and prioritize homelessness assistance. Without careful oversight, these tools can reinforce socioeconomic inequalities.

Before deploying housing AI, cities use AIAs to scrutinize tenant-screening algorithms. These tools often pull data from eviction records, credit histories, and criminal backgrounds. Through an AIA, a municipality can identify if an algorithm unfairly penalizes applicants for past evictions that were dismissed in court, or if it relies on credit metrics that disproportionately exclude minority populations. If the AIA reveals a high risk of discriminatory outcomes, the city will require the software vendor to strip those specific variables from the algorithm before the tool is used to house residents.

Evaluating AI in Public Schools

School districts utilize AI for student lottery assignments, resource distribution, and "early warning systems" designed to predict which students are at risk of dropping out.

When applying an AIA to educational technology, officials focus heavily on student privacy and the risk of self-fulfilling prophecies. For example, if a school district wants to use an algorithm to flag at-risk students, the AIA requires administrators to look at the underlying indicators. If the AI uses neighborhood zip codes or disciplinary records as primary predictors, the assessment might reveal that the tool disproportionately flags low-income or neurodivergent students. The AIA process ensures that these predictive tools are used strictly to route additional support and tutoring to students, rather than tracking them into lower academic tiers.

Evaluating AI in Policing

Law enforcement is the most scrutinized area of municipal AI deployment. Police departments frequently consider tools like predictive policing software, automated license plate readers, and facial recognition technology.

Because the stakes involve citizen liberty and safety, AIAs in policing are rigorous. When evaluating predictive policing software—which forecasts where crimes are likely to occur—an AIA forces the department to confront "dirty data." Historical arrest data often reflects biased policing practices rather than actual crime rates. An AIA highlights how feeding this data into an AI creates a feedback loop, repeatedly sending officers back to the same over-policed neighborhoods.

Furthermore, when evaluating facial recognition, the AIA requires vendors to prove their algorithms have equal accuracy rates across all skin tones, genders, and ages. In many municipalities, the findings from an AIA have led city councils to mandate strict human verification rules—prohibiting arrests based solely on an AI match—or to ban the deployment of the technology entirely.

Governance and Enforcement

Ultimately, an AIA is not just a piece of paperwork; it is a binding governance tool. Municipalities use the final assessment to establish legal guardrails. If a city decides to move forward with an AI tool in housing, schools, or policing, the AIA dictates the terms of the vendor contract. It ensures the government retains the right to audit the proprietary code, mandates regular public reporting on the AI's performance, and establishes clear protocols for shutting the system down if it causes unintended harm to the community.