Artificial Intelligence

Great Power Competition and the AI Revolution: A Range of Risks to Military and Strategic Stability

By Elsa Kania
Tuesday, September 19, 2017, 5:00 PM

Today’s rapid advances in artificial intelligence (AI) could disrupt and destabilize the existing military balance—and not necessarily for the reasons that have captured popular imagination. The potential realization of or “” merits discussion, but it remains a relatively distant possibility. Yet advances in AI have already started to provoke fear, and often hyperbole, about the threats of “,” a looming and the risk of an . Earlier this month Elon Musk even , “Competition for AI superiority at national level most likely cause of WW3 [in my opinion].”

These fears may be premature, but AI’s disruptive potential is real. The recent progress in AI has primarily involved machine learning and particularly deep learning techniques, such as the use of deep neural networks, across disciplines including computer vision, pattern recognition and natural language processing. Since 2016, several critical milestones have revealed the rapid pace of advances and potential real-world applications. In 2016, the of “” in DARPA’s Cyber Grand Challenge the potential of autonomous detection and patching of software vulnerabilities to transform cybersecurity. With the computer program AlphaGo’s historic defeat of Lee Sedol in their 2016 and subsequent over Ke Jie, the world’s top Go player, AI has mastery of the game of Go, which requires strategizing, at least a decade than expected.

Recognizing the disruptive, even revolutionary implications of AI for national defense, the United States, China and Russia are actively seeking to advance their capabilities to employ AI for a range of military applications. In spring 2017, the Department of Defense (DoD) it had established an Algorithmic Warfare Cross-Functional Team “to accelerate DoD’s integration of big data and machine learning.” This summer, China the New Generation AI Development Plan, which to “lead the world” in AI by 2030. This plan calls for military-civil fusion in AI to dual-use advances for applications in national defense, including in support of command decision-making, military deduction, and defense equipment. Meanwhile, the Russian military has been its efforts in intelligent robotics, and Russian President Vladimir Putin , “Whoever becomes the leader in [AI] will become the ruler of the world.” Indeed, the advent of AI in warfare appears to be resulting in a transformation of the character of conflict beyond information-age warfare toward “algorithmic warfare,” in the U.S. military’s phrasing, or “intelligentized” (智能化) warfare, as Chinese military thinkers it. 

Despite recurrent calls to —and a recent that articulated concerns that the development of lethal autonomous weapons would open a “Pandora’s box” and risk unleashing “weapons of terror”—an outright ban is to be . It is improbable that major powers would accept constraints on capabilities considered critical to their future military power. Even attempts to pursue some form of regulation or an international treaty to restrain military applications of AI could be readily overtaken by technological developments. The of this dual-use technology will also be difficult to control.

Consequently, major militaries should take a proactive approach to evaluating and mitigating the potential risks introduced by advances in military applications of AI. This is in their interest, as the U.S., China and Russia still share at least a basic commitment to strategic stability and recognize the undesirability of inadvertent escalation.

To date, much of the analysis and attention associated with the risks of AI in warfare has concentrated on and s associated with the potential use of lethal autonomous weapons systems (LAWS). There has been an international  on the challenges of  and  the law of armed conflict—with core principles of necessity, distinction, proportionality and humanity—to the use of autonomous weapons systems, but it remains unclear how different militaries will interpret and abide by this traditional framework. Concurrently, as Paul Scharre of the Center for a New American Security discussed in a on the topic, the employment of autonomous weapons could result in a range of operational risks, inevitable failure in complex systems, adversary attempts to attack or otherwise undermine autonomous systems (e.g., spoofing and behavioral hacking), or unanticipated interactions between adversarial systems.

These issues are critical and should also inform thinking on the risks that arise out of military applications of AI beyond the context of autonomous weapons. Indeed, risks that might result from current, comparatively uncontroversial military uses of AI also deserve sustained attention. Despite fears that AI is “,” the present limitations of AI, rather than its potential power or questions of controllability, could become problematic in the near term. In some cases, errors and issues might arise from seemingly routine applications, even when the algorithm in question is not used directly in a weapons system or to make a life-and-death decision.

Consider, for example, the implications of rote errors for the U.S. and Chinese militaries’ progression toward using AI for the automation of intelligence, surveillance and reconnaissance (ISR) functions, particularly in support of command decision-making. With , the Defense Department plans to accelerate its integration of big data and machine learning, using computer vision algorithms to the automated processing of data, video and imagery. Through this initiative, the department to have the capacity to “deploy” algorithms in a war zone by the end of 2017. In addition, the U.S. Air Force to leverage AI to process information at speed and scale, through algorithms and human-machine interfaces, and to leaders’ decision-making capabilities. Similarly, the Chinese People’s Liberation Army (PLA) is developing algorithms that enable data fusion, enhance intelligence analysis, and support command decision-making. For instance, the PLA is funding research and development related to target recognition, as well as the processing of sensor data and satellite imagery, based on machine learning. In particular, the Chinese army on the potential of AI in operational command and assistance to decision-making.

These military applications of AI do not directly involve decisions about lethal force but could cause errors that might contribute to crisis instability or exacerbate the risks of escalation. At this stage of development, AI remains far from intelligent, to make mistakes no human would make. Such errors can be unpredictable or difficult to mitigate. In certain cases, the results can be or . In a military context, however, there could be adverse consequences, with higher likelihood of errors or unexpected as the level of complexity increases and if a situation exceeds the expected parameters of an algorithm.

The use of machine learning in support of military intelligence, surveillance and reconnaissance capabilities or command systems could create new avenues for misperception or miscalculation. For instance, if a computer vision algorithm used to process satellite imagery makes an error in identifying a perceived threat or potential target, the result could be a mistake in analysis or even on the battlefield. Similarly, if an algorithm used for machine translation or natural language processing incorrectly renders a critical piece of intelligence, inaccurate information could be introduced into analytical and decision-making processes. (Of course, in some cases, the use of AI could serve to mitigate suboptimal analysis and decision-making that might otherwise arise from human .) There is also the threat that adversaries countermeasures that deliberately damage or interfere with each other’s AI systems, seeking to distort data or target the algorithm itself. 

Looking to another near-term application, there is likely to be a trend toward the use of AI-enabled cyber capabilities by multiple militaries (and even non-state actors), introducing greater degrees of autonomy to a complex and contentious domain. The 2012 Defense Department on “Autonomy in Weapons Systems” explicitly indicated that the policy, which called for the exercise of “appropriate levels of human judgment” over use of force, did “not apply to autonomous or semi-autonomous cyberspace systems for cyberspace operations.” The sheer speed required in certain cyber operations, as in , could necessitate greater degrees of autonomy. In August, the Defense Department officially the technology associated with Mayhem, which will be used to automate the detection and patching of software vulnerabilities. Although Mayhem is intended for a defensive mission, similar techniques could be and for offensive purposes. The Chinese military is also likely to look to AI to enhance its offensive and defensive cyber capabilities, including under the aegis of its new . The trend towards integrating AI with cyber capabilities to achieve an advantage could exacerbate escalatory dynamics in this contested domain, particularly if such capabilities start to proliferate.

Looking forward, AI will enable disruptive military capabilities while creating systemic risks to military and strategic stability. Now is the time to start considering the potential ramifications of these technological trends and evaluating appropriate parameters that might mitigate risks of error or escalation. To start, great-power militaries might consider pursuing—and perhaps, as a future confidence-building measure, committing to—an initial set of pragmatic measures, such as:

  • Engaging in robust testing of the safety and integrity of military AI systems, focusing on potential errors or failures that might occur in open, uncontrolled environments;
  • Creating redundancies in military AI systems, including those used to support intelligence, surveillance and reconnaissance capabilities, to enable verification, such that there are multiple methods to detect errors and evaluate outputs, in order to ensure consistency with actual ground truth;
  • Exploring options for fail-safe measures or “” to allow for de-escalation or de-confliction in the case of unintended engagements or escalation; and
  • Ensuring the “” of AI systems to lessen issues of compromised decision-making, while perhaps ensuring informed, “” whenever feasible.