Can a probability be fair?

I recently watched Justicia artificial, by Simón Casal. A very simple scene stayed with me.

A judge has to decide a case involving one person. She listens, observes and tries to understand. Beside her, a screen does something else: it calculates.
82% probability of reoffending.^[1]

And I stopped there.

I admit that, watching some human decisions, I sometimes think—half joking, half seriously—that perhaps robots would govern us better. They do not get tired, have a bad day or act on personal sympathies. They can process amounts of information we cannot.

It sounds tempting.

Until that 82% is facing a person.

The film imagines a near future in which judges might be replaced by an AI system promising justice that is faster, more objective and less influenced by human interests.

I like the question because its answer is not as comfortable as it first seems.

People have biases. We interpret. We make mistakes. Emotions, prejudice and interests can influence us.

But we also do something hard to put into a formula: we look at context.

A probability of reoffending may be built from thousands of data points and be statistically strong. But it still describes people who resemble this person. Not necessarily this person.

So what do we do with the remaining 18%?

Empathy enters the picture. So does the problem.

Empathy can help us understand what a pattern misses. It can also lead us astray. Feeling more does not always mean deciding better.

Perhaps the film left me thinking less about humans versus machines and more about which part of a decision we are willing to delegate.

AI can help identify patterns, retrieve case law, compare cases or estimate risks. In California, risk assessments may inform a court, but they are not meant to replace judicial judgement or serve as the sole basis for a decision.^[2]

But helping us decide and deciding for us are different things.

I do not know exactly where the boundary lies.

I suspect that the greater the impact a decision has on someone’s life, the more important it is to know who is accountable for it.

An algorithm can calculate a probability. The difficult part starts just after that.