The COMPAS Controversy: AI Bias in Criminal Justice

The Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) algorithm became the focal point of a global debate on artificial intelligence ethics after a 2016 investigation revealed significant racial disparities in its error rates. Used by U.S. courts to predict the likelihood of a defendant reoffending, the tool was found to falsely flag Black defendants as future criminals at nearly twice the rate of white defendants. This discovery ignited a fierce controversy among data scientists, legal experts, and civil rights advocates regarding the fairness of predictive policing, the mathematical definitions of bias, and whether automated risk assessments belong in the criminal justice system.

COMPAS was developed by the private company Northpointe (now Equivant) to help judges make more objective decisions regarding bail, sentencing, and parole. By evaluating a defendant's answers to 137 questions—none of which explicitly ask about race—the software generates a recidivism risk score from one to ten. The goal was to replace human judicial bias with data-driven objectivity.

In May 2016, the investigative journalism organization ProPublica analyzed the COMPAS scores of more than 7,000 people arrested in Broward County, Florida, over a two-year period and checked to see how many were charged with new crimes later on. Their analysis showed that the algorithm's overall accuracy was roughly equivalent across racial groups. However, when the algorithm got things wrong, it failed differently for Black and white defendants.

The core of the controversy lay in the disparate false-positive and false-negative rates:

• False Positives: Black defendants who did not reoffend were falsely labeled as "high risk" at a rate of 44.9%, compared to just 23.5% for white defendants.
• False Negatives: White defendants who did go on to commit new crimes were mistakenly labeled as "low risk" 47.7% of the time, compared to 28.0% for Black defendants.

Northpointe disputed ProPublica’s findings, arguing that COMPAS was fair because it achieved "predictive parity." This meant that a specific risk score meant the same thing regardless of race; a score of seven indicated roughly the same probability of recidivism whether the defendant was Black or white.

This clash exposed a fundamental mathematical dilemma in artificial intelligence. Computer scientists subsequently proved that when two groups have different baseline rates of arrests—often a result of historical socioeconomic factors and systemic policing disparities—it is mathematically impossible for an algorithm to simultaneously achieve predictive parity (Northpointe's standard) and equal false-positive rates (ProPublica's standard).

Even though race was excluded from the questionnaire, the algorithm utilized factors such as arrest history, employment status, education, and zip code. Critics pointed out that in a society with systemic inequalities, these data points act as proxies for race. By training an AI on historical crime data, the system inevitably learns and replicates existing biases in law enforcement.

Furthermore, the debate highlighted the issue of transparency in the legal system. Because COMPAS is a proprietary commercial product, its underlying source code and how it weighs different factors are trade secrets. Defense attorneys could not fully scrutinize how a client's risk score was calculated, raising serious constitutional questions about the right to due process.

Ultimately, the COMPAS controversy proved that algorithms are not inherently neutral. It transformed the field of machine learning by establishing "algorithmic fairness" as a critical area of study and forcing courts across the United States to reconsider their reliance on opaque, automated decision-making tools when human liberty is at stake.