In a near-future war\u2014one that might begin tomorrow, for all we know\u2014a soldier takes up a shooting position on an empty rooftop. His unit has been fighting through the city block by block. It feels as if enemies could be lying in silent wait behind every corner, ready to rain fire upon their marks the moment they have a shot.<\/p>\n
Through his gunsight, the soldier scans the windows of a nearby building. He notices fresh laundry hanging from the balconies. Word comes in over the radio that his team is about to move across an open patch of ground below. As they head out, a red bounding box appears in the top left corner of the gunsight. The device\u2019s computer vision system has flagged a potential target\u2014a silhouetted figure in a window is drawing up, it seems, to take a shot.<\/p>\n<\/div>\n<\/div>\n
The soldier doesn\u2019t have a clear view, but in his experience the system has a superhuman capacity to pick up the faintest tell of an enemy. So he sets his crosshair upon the box and prepares to squeeze the trigger.<\/p>\n
In different war, also possibly just over the horizon, a commander stands before a bank of monitors. An alert appears from a chatbot. It brings news that satellites have picked up a truck entering a certain city block that has been designated as a possible staging area for enemy rocket launches. The chatbot has already advised an artillery unit, which it calculates as having the highest estimated \u201ckill probability,\u201d to take aim at the truck and stand by.<\/p>\n
According to the chatbot, none of the nearby buildings is a civilian structure, though it notes that the determination has yet to be corroborated manually. A drone, which had been dispatched by the system for a closer look, arrives on scene. Its video shows the truck backing into a narrow passage between two compounds. The opportunity to take the shot is rapidly coming to a close.<\/p>\n
For the commander, everything now falls silent. The chaos, the uncertainty, the cacophony\u2014all reduced to the sound of a ticking clock and the sight of a single glowing button:<\/p>\n
\u201cAPPROVE FIRE ORDER.\u201d <\/strong><\/div>\n It should be of little surprise, then, that states and civil society have taken up the question of intelligent autonomous weapons\u2014weapons that can select and fire upon targets without any human input\u2014as a matter of serious concern. In May, after close to a decade of discussions, parties to the UN\u2019s Convention on Certain Conventional Weapons agreed, among other recommendations, that militaries using them probably need to \u201climit the duration, geographical scope, and scale of the operation\u201d to comply with the laws of war. The line was nonbinding, but it was at least an acknowledgment that a human has to play a part\u2014somewhere, sometime\u2014in the immediate process leading up to a killing.<\/p>\n But intelligent autonomous weapons that fully displace human decision-making have (likely) yet to see real-world use. Even the \u201cautonomous\u201d drones and ships fielded by the US and other powers are used under close human supervision. Meanwhile, intelligent systems that merely guide the hand that pulls the trigger have been gaining purchase in the warmaker\u2019s tool kit. And they\u2019ve quietly become sophisticated enough to raise novel questions\u2014ones that are trickier to answer than the well-\u00adcovered wrangles over killer robots and, with each passing day, more urgent: What does it mean when a decision is only part human and part machine? And when, if ever, is it ethical for that decision to be a decision to kill?<\/p>\n For a long time, the idea of supporting a human decision by computerized means wasn\u2019t such a controversial prospect. Retired Air Force lieutenant general Jack Shanahan says the radar on the F4 Phantom fighter jet he flew in the 1980s was a decision aid of sorts. It alerted him to the presence of other aircraft, he told me, so that he could figure out what to do about them. But to say that the crew and the radar were coequal accomplices would be a stretch.<\/p>\n That has all begun to change. \u201cWhat we\u2019re seeing now, at least in the way that I see this, is a transition to a world [in] which you need to have humans and machines \u2026 operating in some sort of team,\u201d says Shanahan.<\/p>\n The rise of machine learning, in particular, has set off a paradigm shift in how militaries use computers to help shape the crucial decisions of warfare\u2014up to, and including, the ultimate decision. Shanahan was the first director of Project Maven, a Pentagon program that developed target recognition algorithms for video footage from drones. The project, which kicked off a new era of American military AI, was launched in 2017 after a study concluded that \u201cdeep learning algorithms can perform at near-\u00adhuman levels.\u201d (It also sparked controversy\u2014in 2018, more than 3,000 Google employees signed a letter of protest against the company\u2019s involvement in the project.)<\/p>\n With machine-learning-based decision tools, \u201cyou have more apparent competency, more breadth\u201d than earlier tools afforded, says Matt Turek, deputy director of the Information Innovation Office at the Defense Advanced Research Projects Agency. \u201cAnd perhaps a tendency, as a result, to turn over more decision-making to them.\u201d<\/p>\n Another gunsight, built by the company Smartshooter, is advertised as having similar capabilities. According to the company\u2019s website, it can also be packaged into a remote-controlled machine gun like the one that Israeli agents used to assassinate the Iranian nuclear scientist Mohsen Fakhrizadeh in 2021.<\/p>\n Decision support tools that sit at a greater remove from the battlefield can be just as decisive. The Pentagon appears to have used AI in the sequence of intelligence analyses and decisions leading up to a potential strike, a process known as a kill chain\u2014though it has been cagey on the details. In response to questions from\u00a0MIT Technology Review,\u00a0Laura McAndrews, an Air Force spokesperson, wrote that\u00a0the service \u201cis utilizing a human-\u00admachine teaming approach.\u201d<\/p>\n The range of judgment calls that go into military decision-making is vast. And it doesn\u2019t always take artificial super-intelligence to dispense with them by automated means.<\/strong><\/p>\n<\/blockquote>\n Other countries are more openly experimenting with such automation. Shortly after the Israel-Palestine conflict in 2021, the Israel Defense Forces said it had used what it described as AI tools to alert troops of imminent attacks and to propose targets for operations.<\/p>\n<\/div>\n<\/div>\n The Ukrainian army uses a program, GIS Arta, that pairs each known Russian target on the battlefield with the artillery unit that is, according to the algorithm, best placed to shoot at it. A report by The Times, a British newspaper, likened it to Uber\u2019s algorithm for pairing drivers and riders, noting that it significantly reduces the time between the detection of a target and the moment that target finds itself under a barrage of firepower. Before the Ukrainians had GIS Arta, that process took 20 minutes. Now it reportedly takes one.<\/p>\n Russia claims to have its own command-and-control system with what it calls artificial intelligence, but it has shared few technical details. Gregory Allen, the director of the Wadhwani Center for AI and Advanced Technologies and one of the architects of the Pentagon\u2019s current AI policies, told me it\u2019s important to take some of these claims with a pinch of salt. He says some of Russia\u2019s supposed military AI is \u201cstuff that everyone has been doing for decades,\u201d and he calls GIS Arta \u201cjust traditional software.\u201d<\/p>\n The range of judgment calls that go into military decision-making, however, is vast. And it doesn\u2019t always take artificial super-\u00adintelligence to dispense with them by automated means. There are tools for predicting enemy troop movements, tools for figuring out how to take out a given target, and tools to estimate how much collateral harm is likely to befall any nearby civilians.<\/p>\n None of these contrivances could be called a killer robot. But the technology is not without its perils. Like any complex computer, an AI-based tool might glitch in unusual and unpredictable ways; it\u2019s not clear that the human involved will always be able to know when the answers on the screen are right or wrong. In their relentless efficiency, these tools may also not leave enough time and space for humans to determine if what they\u2019re doing is legal. In some areas, they could perform at such superhuman levels that something ineffable about the act of war could be lost entirely.<\/p>\n Eventually militaries plan to use machine intelligence to stitch many of these individual instruments into a single automated network that links every weapon, commander, and soldier to every other. Not a kill chain, but\u2014as the Pentagon has begun to call it\u2014a kill web.<\/p>\n In these webs, it\u2019s not clear whether the human\u2019s decision is, in fact, very much of a decision at all. Rafael, an Israeli defense giant, has already sold one such product, Fire Weaver, to the IDF (it has also demonstrated it to the US Department of Defense and the German military). According to company materials, Fire Weaver finds enemy positions, notifies the unit that it calculates as being best placed to fire on them, and even sets a crosshair on the target directly in that unit\u2019s weapon sights. The human\u2019s role, according to one video of the software, is to choose between two buttons: \u201cApprove\u201d and \u201cAbort.\u201d<\/p>\n Let\u2019s say that the silhouette in the window was not a soldier, but a child. Imagine that the truck was not delivering warheads to the enemy, but water pails to a home.<\/p>\n<\/div>\n<\/div>\n![\"\"](\"https:\/\/www.fie.undef.edu.ar\/ceptm\/wp-content\/uploads\/2023\/08\/Comp-2-771x1024.webp\"){kind=spacer}
\n
\n
![\"\"](\"https:\/\/www.fie.undef.edu.ar\/ceptm\/wp-content\/uploads\/2023\/08\/Comp-1-771x1024.webp\"){kind=spacer}