The Case for Banning Algorithmic Sentencing Tools

In 2013, a judge in Wisconsin sentenced a man named Eric Loomis to six years in prison. The judge cited many factors. One of them was a number generated by a computer program that Loomis had never seen, could not examine, and had no legal right to challenge. The company that built the program called its methodology a trade secret. The Wisconsin Supreme Court upheld the sentence. [1]

This is not an isolated incident. It is standard practice in courts across the United States.

Algorithmic risk assessment tools, software programs that claim to predict the likelihood that a defendant will reoffend, are currently used in sentencing decisions, bail determinations, and parole hearings in jurisdictions across the country. The most widely used of these tools is called COMPAS. ProPublica investigated it in 2016 and found something the tools' manufacturers had not advertised: COMPAS was nearly twice as likely to falsely flag Black defendants as high risk compared to white defendants. [2] A 2019 study published in Science Advances confirmed the bias. [3]

These tools are still in use today.

Mind on Trial's position is straightforward: algorithmic risk assessment tools should be banned from criminal sentencing until they can meet basic standards of transparency, accuracy, and racial equity. Here is why.

The transparency problem

The most fundamental problem with algorithmic sentencing tools is that nobody can see inside them. COMPAS and tools like it are proprietary products built by private companies. Their methodologies are protected as trade secrets. Defense attorneys cannot examine the algorithm. Defendants cannot challenge the inputs. Judges often do not understand how the score was calculated. They simply receive a number and a risk category and are expected to incorporate it into a sentencing decision that will determine years of a person's life.

This violates a basic principle of due process. In American law, a defendant has the right to confront the evidence against them. A secret algorithm that generates a risk score based on undisclosed inputs and an unverifiable methodology is not confrontable. It is a black box with the power of a witness and none of the accountability. [4]

The accuracy problem

Risk assessment tools claim to predict future behavior. This is an inherently uncertain enterprise. A 2018 study published in Science found that neither COMPAS nor simple two-variable models performed significantly better than untrained humans at predicting recidivism. [5] The tools do not dramatically outperform human judgment. They simply present their predictions with the false authority of a number.

Numbers feel objective. They are not. Every algorithm reflects the assumptions, data, and priorities of the people who built it. When those inputs contain historical patterns of racial bias in policing and prosecution, the algorithm learns and replicates those patterns. The bias is not a bug in the system. It is a feature of any system trained on historically biased data.

The racial equity problem

The ProPublica investigation found that COMPAS assigned higher risk scores to Black defendants than white defendants with similar criminal histories and demographic profiles. Black defendants were nearly twice as likely to be falsely labeled high risk. White defendants were nearly twice as likely to be falsely labeled low risk. [2]

This is not a minor calibration issue. It is a systematic distortion that results in Black defendants receiving longer sentences for equivalent conduct. Algorithmic bias compounds existing disparities in a system already documented to produce racially unequal outcomes at every stage from arrest to conviction to sentencing.

Incorporating a racially biased algorithm into the sentencing process does not make sentencing more objective. It makes it more efficiently biased.

What we are calling for

Mind on Trial supports the following specific reforms.

First, a moratorium on the use of proprietary algorithmic risk assessment tools in criminal sentencing, bail, and parole decisions until full transparency standards are met. No tool whose methodology cannot be examined by defense counsel should be used to influence a sentencing decision.

Second, mandatory independent audits for any risk assessment tool used in the criminal justice system, with results published publicly and updated annually.

Third, a federal prohibition on the use of race-correlated variables as proxy inputs in risk assessment algorithms. Tools that use zip code, employment history, or family criminal history as inputs are using variables that correlate strongly with race and reproduce racial disparities under the cover of neutrality.

Fourth, a requirement that any risk assessment score presented to a judge be accompanied by a plain-language explanation of what factors produced the score and what its documented error rates are, including its false positive rates broken down by race.

The bottom line

Algorithms are not neutral. They are not objective. They are not a solution to the problem of bias in sentencing. They are a mechanism for encoding and automating bias at scale, while making it harder to see and harder to challenge.

Eric Loomis could not challenge the score that helped determine his sentence. He did not know what questions it asked, how it weighted the answers, or why it reached the conclusion it did. That is not justice. It is not even the appearance of justice. It is the laundering of a judgment through a machine.

The law should catch up to what the science already knows: predicting human behavior is uncertain, historical data is biased, and no algorithm deserves the trust our courts are placing in it.

If you want to go deeper: ProPublica's original COMPAS investigation is at propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing. The 2019 Science Advances study is available at advances.sciencemag.org.

Sources

[1] State v. Loomis, 881 N.W.2d 749 (Wis. 2016). law.justia.com

[2] Angwin, J., Larson, J., Mattu, S., and Kirchner, L. (2016). Machine Bias. ProPublica. propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

[3] Dressel, J., and Farid, H. (2018). The Accuracy, Fairness, and Limits of Predicting Recidivism. Science Advances, 4(1). advances.sciencemag.org

[4] Pasquale, F. (2015). The Black Box Society: The Secret Algorithms That Control Money and Information. Harvard University Press. hup.harvard.edu

[5] Stevenson, M. T., and Doleac, J. L. (2019). Algorithmic Risk Assessment in the Hands of Humans. IZA Discussion Paper No. 12853. ssrn.com