Ethical Problems around Group Action in the Context of National Security

My speaking script for a panel at the USSOCOM Sovereign Challenge Program 2019 Annual Conference (a CMU-SOCOM panel on ethics, the law of war, and national-security policy), Pittsburgh, spring 2019. Lightly formatted from the original.

Preamble: Ethical Thinking and Acting in a Pluralist World

As a researcher, I have found it both interesting and perilous to make ethical pronouncements in the context of research. There is always the nagging question: “on what authority do I get to will that my ethical values are privileged and should dominate a public decision?” This conundrum has become more salient as my research increasingly touches on questions of value alignment in AI and technology broadly¹.

Luckily, I am not the first researcher to worry about this. Max Weber² makes an interesting argument for what the role of a “scientist”³ should be in addressing such questions: If we grant the premise that “science” cannot determine how we should live & act, then the role of science in ethical thinking is primarily as a tool to see more clearly about the necessary or probable outcomes of our ethical choices. Basically, to provide an “Archimedean point” (as much as such a thing is possible), from which we can better understand our ethical dilemmas.

The role is emphatically not to thumb the scales in favor of personal ethical preferences. This is the standard I will try to model in the following discussion. Partly because my ethical views on the topic are not set in stone. But mostly because this perspective seems more constructive and/or engaging for achieving any form of consensus.

Why should one avoid proceeding from an “ethics of conviction” in this domain? Well… if we take Nietzsche’s “God is dead” maxim seriously, we would need to confront the deep-seated condition of value pluralism that circumscribes our social and political lives. To live is to act. And we act constantly against the background fact that there is a “plurality of [often] incompatible orientations to life.”

One could argue that the fact of value pluralism makes a strong case for emphasizing individual liberty in how we conduct our lives. Indeed, Isaiah Berlin argues as such. But pluralism has thorny implications for group action and collective choice. Plural, divergent individual values and rationalities make the problem of coordinated action in non-authoritarian systems… difficult.

In this discussion, I will try to focus on two ethical issues that arise when we aim to deploy tech to achieve the collective social good of national security:

• Doctrinal diversity at the global-level
• Value disagreements underpinning the Civilian-Military divide

I want to frame these issues as instances of the problem of group action under value pluralism.

The Operating Conditions

It is useful to take stock of the conditions under which we are working… Let us label international concerns as “external” and the intra-state concerns as “internal.” The external conditions around the use of technology in war include the following:

• [Emergence of Tech Near-peers]: The recent flurry of national AI strategies and talk arms races may have been overblown. But they highlight the fact that the US has tech-competent powers to compete against. And some of these powers have geopolitical interests to pursue against the US. In the field of AI specifically, we have seen China emerge as a very energetic and ambitious peer in terms of investment and patents. And China’s deployment trajectory has already seen them gain supremacy in computer vision & facial recognition AI systems. Russia is also an important player. It is especially interesting to note their competence in cyber and influence operations.
• [Lower barriers to entry]: Specifically for AI, the barriers to the effective deployment of tech are not as high. Loosely speaking, some of the primary factors for entry are: data ecosystems, compute/hardware infrastructure, algorithms/code, and technical expertise. Data and compute are increasingly easy to deploy globally. AI technical expertise is rather cosmopolitan. And much of the relevant code/algorithms is publicly accessible. It would be interesting to observe how non-state actors (NSAs) respond to lower barriers to access. We already see NSAs using drones in attacks.
• [Dual-use]: Technology is often worse than dual-use. This is nothing more than a statement of the fact the problems to which tech innovations are deployed is strongly determined the deploying agent’s culture, values, specific problems, etc. This is evident in the differences in use of CV/FRT in China-vs.-the USA. The American killer-app for CV seems to be unlocking phones or tagging social media pictures… The Chinese killer-app seems to be social control and surveillance. This concern highlights the disconnect between tech use and the intentions controlling tech development. Arguably, this phenomenon is fundamental to our nature as a tool-using species.

Let’s focus on the following internal conditions:

• [Commercially Driven Innovation]: Government R&D funding is a strong feature of the history of AI innovation in the US. This funding monoculture was arguably a strong contributing factor to the occurrence of past AI Winters. But this cycle of AI innovation is largely driven by commercial companies especially in Silicon Valley. This is interesting because the fruits of commercial innovation may no longer translate as readily to national security needs. National AI strategies and funding agencies may be useful for addressing this disconnect. Somewhat…
• [Civilian-Military Disconnect]: One notable trend in the US is the extent to which tech workers and companies espouse strong anti-war sentiments. We see this in the backlash to project Maven and Amazon’s face recognition project. The international nature of the tech workforce, the multinational nature of tech firms, the dominance of commercial R&D funding sources all act to deepen the schism between the civilian and military spheres. A deep Civilian-Military disconnect has implications for the ability of a non-authoritarian government to deploy technology in the service of national security. Especially when the private sector outstrips the government’s tech expertise.

Highlighting Pluralist Themes and Implications?

I will try to use the rest of the discussion to highlight two clusters of questions that need “scientific” attention.

• [International Doctrinal Diversity]: We can observe that different cultures or jurisdictions adopt and use technology differently. In fact, in a certain sense, the world is fractured into different “technocultures” determined by how local cultures and regulations determine local tech use⁴. Again, accommodating some amount of pluralism in how individual regimes self-determine on tech is important, coming from a liberal system of values⁵. There are a number of interesting implications to this framing. But the one of particular interest to me starts with the understanding that peace and security is a common/social good that relies on group action at the international level. But a diversity in local doctrines of tech use can undermine such group action in subtle ways. Think for example about the USA’s stated doctrine of requiring meaningful human oversight in autonomous weapons. Imagine a scenario in which a near-peer is less reticent about deploying lethal autonomous weapons⁶.
  • What are the deterrence implications or escalation dynamics between competitors with different doctrines of use?
  • What about competition with non-state actors with whom alignment is not possible and no diplomatic channels exist for reasoning/de-escalating?
  • If agreements & treaties are viable, what does the verification of autonomy levels look like?
• [Navigating the Civilian-Military Divide]: In the US, there seems to be significant value disagreement between stakeholders in civilian and military spaces. Many tech workers are espousing and acting on strong pacifist inclinations. There are implications for the state’s ability to credibly defend its interests in an increasingly multi-polar and technologically-competitive world. Presumably both civilian and military stakeholders share the value of preferring peace and security to war.
  • What effective mechanisms can we bring to bear for achieving useful consensus ethical group action when stakeholders value the same ends but disagree on means⁷?