AI and the Equitable Society

My speaking script for a spotlight talk at the K&L Gates-CMU Conference on Ethics and AI, April 2018. Lightly formatted from the original.

Prologue: Historical Echoes

Today, we are talking about equity and artificial intelligence. Or more concretely, algorithmic decision-making artifacts.

I would like to start the conversation by looking to antiquity, to two great Athenians: Plato and Socrates. I will be arguing that the contrast between these two men prefigures the complexities we will be exploring today.

I have one key take-away for this conversation: “Participatory Equity.” My goal is to convince you that this concept should be more central to our work. I contend that it is the foundation of any successful infrastructure for equity in algorithmic decision-making. Now you can take a nap without missing much…

Back to Plato… The mental picture I have of Plato is of a genius. But also of a man deeply traumatized by one pivotal event: the state-sanctioned execution of his teacher and mentor Socrates. His scars run so deep he spends the rest of his life trying to compensate by capturing (and probably modifying) the wisdom of Socrates in writing. The result is a blessed legacy of written philosophy which his student Aristotle continued. The West owes a lot to this philosophical tradition.

There is a darker Platonic legacy I’d like to focus on today. Plato reacts to Socrates’ death by retreating from the active life, the agora, the marketplace of ideas and plural perspectives. He sets up the archetype for the solitary contemplative philosopher. He dreams up supposed utopias without engaging and accounting for the plural perspectives of those who would be subjects in that utopia. One can almost imagine Plato, positively dripping with disdain for the demos (people) he holds responsible for the death of Socrates, as he suggests that the perfect state is one ruled by an aloof philosopher king.

Contrast this with Socrates’ style of inquiry. Socrates engages deeply and critically with his community when he aims to answer the ‘simple’ question: “What is Wisdom?” He may not have found his answer. But, at the very least, he can claim some defense against the biases and delusions his singular personal perspective brings.

Now imagine how these two greats may have approached the questions that lie before us: What is Equity? How do algorithms interact with equity of impact and access?

Act I: AI Equity?

A few months ago, I was introduced at a gathering as “an engineer of fairness.” Cute but a little paradoxical, no? Engineers stereotypically evokes images of sober-faced individuals, given to precise “objective” analysis and design. Fairness or equity, on the other hand, is this relative, fuzzy, subjective, and contextual thing; notoriously slippery to capture. Any exploration of equity in AI is doomed to demonstrate this “compresence of opposite” qualities.

Exploring AI equity used to be a dubious enterprise. You were likely to get dismissive remarks like “what is equity?” [in the style of Pontius Pilates’ question “what is truth?”] Up until recently, typical policy-oriented conversations on AI were… heavy on drama… These have been fun speculative conversations filled with references to “killer sentient robots,” existential AI threats, and dysfunctional pin-making AI factories.

One might argue that these concerns are relevant but possibly over-blown. More pointedly, maybe the focus on those lines of inquiry is the product of a privileged perspective.

These arguments are probably lazy. So let me attempt a reconciliation by suggesting that AI equity and existential risks concerns may actually have common roots. For example, both are related to questions of value alignment and explainability.

Value Alignment: How do we design algorithmic systems that align with societal values like fairness? Explainability: in the interest of procedural transparency, how do we derive meaningful stable explanations from algorithmic systems?

When we focus on equity, there are definitional issues and perceived arbitrariness. But we are familiar with examples of algorithms that violate some implicitly or explicitly defined equity norm. These examples have served to challenge that “deep slumber of settled opinion” that ascribes objectivity or infallibility to artificial agents. Recently Luciano Floridi writes [uncontroversially]:

“it is now commonly agreed that [technological] design […] is never ethically neutral but always embeds some values, whether implicitly or [explicitly].”

And the question of AI equity turns on how much those values align with societal norms.

Speaker’s note (spoken live, not in the written script): More concretely… [Discuss the 3 sample cases; Discuss the framing on how AI design might fall short of equity]

Speaker’s note (spoken live): [Discuss CJ & Ins scenarios in detail; focus on normative set-up]

Act II: From Particulars, Generalities

We can use these examples to draw key principles and trends that underpin equity.

The first is the rather obvious point that equity norms are necessarily context-specific. Context determines the relevant norm. And the relevant context is often fuzzy or fluid. This simple point already deflates any hope for a prescriptive universal definition for equity; the sort of definition a Plato-like philosopher might hope to discover in intense solitary contemplation. Incidentally, also the sort of prescriptive definition a designer might hope to discover and implement to make “fair algorithms.”

The second point builds on the contextual nature of equity norms. Let’s call this the inconsistency of prescriptive equity norms. It’s useful to think of two sides on this. Chouldechova and others highlight what we might call a set of weak impossibility theorems: Pick any 3 prescriptive equity norms for an algorithmic risk assessment tool (e.g. the triad: group calibration, equal false positive rates, equal false negative rates). Then it is generally impossible to satisfy these norms simultaneously. Narayanan points out this inconsistency result extend to other math definitions of fairness (discussed under the label “21 incompatible definitions of fairness”).

On hearing this, you might decide “well, since I can’t have it all, I’ll just have my users choose or prioritize which norms to focus on.” Unfortunately we run into strong impossibility theorems (by Arrow, Gibbard, Sattherthwaite) that thwart such a democratic response. The general gist of these theorems is to point out the difficulty (or impossibility) of reasonably aggregating collective normative preferences.

The third point is a framing I’ve found increasingly useful. Many distributive/allocative programs seem to fall on a spectrum between entitlement and market-based distributive problems.

Entitlement programs refer to settings where there exist clear equity norms governing the allocation decision. Normative clarity usually involves the existence of governing laws/regulations. And the object of distribution is often a public good subject to some indefeasible right. US criminal justice is one example of such an allocative problem: dispensing the public good of law-enforcement subject to constitutional norms, particularly the 14th Amendment norm of equal protection.

A few questions arise here: Are the governing equity norms necessarily equitable? [“Legality is about power, not justice”] And if they are not, what remedies are available? Social movements like Black Lives Matter are challenges to the fairness of governing norms.

Market programs are settings where distributive decisions are based just on matching demand to supply, subject to little or no other equity considerations.

Most allocation programs are rarely one or the other. Examples that lean towards the market end include insurance, employment, housing.

Adopting equity regulations in market-heavy programs may sometimes imposes a trade-off between equity and market efficiency. This usually leads to a type of dead-weight loss: the cost of fairness.

One might ask the question: how large are these costs and who should bear them? We see these questions play out in controversies over cost-sharing setups like:

• universal health coverage
• auto insurance coverage.

Things get extra-interesting when you throw in the use of genetic or other personalized risk-relevant information.

Act III: A More Positive Infrastructure for Equity

OK I feel like I’ve been full of bad news about algorithmic equity. Let’s see if we can construct a more positive vision from the rubble.

We have poked some holes in the idea of prescriptive standards of fair algorithms. And even if some ideal standard existed, it might be hard to retain the active consent/buy-in of our user base.

This suggests that algorithmic equity requires a more participatory design model. A model that constantly examines the plural perspectives of people affected by a decision-making artifact.

When we make arguments for more diversity in the ranks of algorithm and platform designers, it is precisely because diversity engages more societal perspectives into the design process. But diversifying the designer class is insufficient. Algorithms will always affect the lives of people who cannot possibly be represented in that class.

More pointedly, Equity concerns are a distributed form of knowledge; no single controlling group can efficiently coordinate the transmission and accommodation of such knowledge.

Basically we need fewer Platos designing from the isolation of Silicon Valley. We need more Socrates going out to engage far-flung subjects. It’s messier work. Luckily we have a few eager pioneers. I am reminded of Chouldechova’s recent work addressing equity concerns with the Allegheny County child welfare system.

A participatory model emphasizes the process nature of equity. Equity is unlikely to be some fixed implementation point. What we deem equitable is just an unstable agreement that will shift as norms, experience, and affected parties change. Judge Learned Hand expressed this process understanding of equity:

“Justice/Equity is the tolerable accommodation of the conflicting interests of society.”

We are doomed to work on shifting normative sands. So maybe we should focus more on the design of an infrastructure for addressing AI concerns.¹ Such an infrastructure must coordinate and respond to equity concerns robustly. What would be a minimal specification for such an infraethics? I’d argue for at least the following:

Institutional Fluency: we need more fluency in the principles of equity. We need sophistication in the ways we discuss equity principles. Beyond just common intuition. Why is this important? As Peyton Young put it:

“Equity principles are the instruments by which societies resolve distributive problems…” [when efficiency yields [unsatisfactory] results]

Lack of fluency in equity principles limits our ability to adjudicate and respond effectively to equity concerns. Fluency requires many of us to reach further and deeper into unfamiliar disciplines; for example, into the older work of economists like Sen, Young, & Shapley.

Avenues & Right of Dissent: Equity concerns are necessarily decentralized. Any implementation process that does not enable meaningful effective dissent leaves the door open to some form of unaddressed inequity. Being more transparency about governing equity norms is an important part of this factor.

Accountable Institutional Redress²: We need mechanisms and incentives that encourage institutions to make & keep promises to remedy when confronted with equity concerns. The history of commercial responses to data breaches suggests that this necessarily involves significant financial penalties.

Epilogue: Provocations on the Cutting-Room Floor

My focus in this conversation is by no means exhaustive. I’m biased to examine allocative/distributive problems. Possibly because these are easier for me to grapple with at a policy shop. There are also several more systemic equity concerns.

Kate Crawford highlighted the equity-related questions of representational harms (in contrast to allocative harms). What are the endemic long-term consequences of poor representation in AI outcomes? What happens if our AI systems continue to learn to represent a CEO as most likely male? Even if these systems accurately represent the current state of the world. And how should we respond to the fact that these poorly representative algorithms are shaping our preferences in unprecedented ways?

Relatedly, there is the question of the interplay between data privacy and equity. Access to privacy-preservation is highly non-uniform. That means that representation in AI-training sets is also non-uniform. How do we tease apart the algorithmic decision-making implications in applications like credit access, welfare systems, employment, etc.? Does more privacy enhance fairness?

More generally, what incentives do algorithmic allocation tools create?

Perhaps others have better answers to these...