Like human troops, the machines took turns covering each other as they advanced. One robot would find a safe spot, stop, and launch the tethered mini-drone<\/a> it carried to look over the next ridgeline while the other bot moved forward; then they\u2019d switch off.<\/p>\n Their objective: a group of buildings on the Army\u2019s Yuma Proving Ground, a simulated town for urban combat training. As one robot held back to relay communications to its distant human overseers, the other moved into the town \u2013 and spotted \u201cenemy\u201d forces. With human approval<\/a>, the robot opened fire<\/a>.<\/p>\n Then the robot\u2019s onboard Aided Target Recognition (ATR) algorithms identified another enemy, a T-72 tank. But this target was too far away for the robot\u2019s built-in weapons to reach. So the bot uploaded the targeting data to the tactical network<\/a> and \u2013 again, with human approval \u2013 called in artillery support.<\/p>\n \u201cThat\u2019s a huge step, Sydney,\u201d said Brig. Gen. Richard Ross Coffman<\/a>, the Project Convergence exercise director. \u201cThat computer vision\u2026 is nascent, but it is working.\u201d<\/p>\n Algorithmic target recognition and computer vision are critical advances over most current military robots, which aren\u2019t truly autonomous but merely remote-controlled: The machine can\u2019t think for itself, it just relays camera feeds back to a human operator, who tells it exactly where to go and what to do.<\/p>\n That approach, called teleoperation, does let you keep the human out of harm\u2019s way, making it good for bomb squads and small-scale scouting. But it\u2019s too slow and labor-intensive to employ on a large scale. If you want to use lots of robots without tying down a lot of people micromanaging them, you need the robots to make some decisions for themselves \u2013 although the Army emphasizes that the decision to use of lethal force will always be made by a human.<\/p>\n So Coffman, who oversees the Robotic Combat Vehicle<\/a> and Optionally Manned Fighting Vehicle<\/a> programs, turned to the Army\u2019s Artificial Intelligence Task Force<\/a>\u00a0at Carnegie Mellon University<\/a>. \u201cEight months ago,\u201d he told me, \u201cI gave them the challenge: I want you to go out and sense targets with a robot \u2014 and you have to move without using LIDAR.\u201d<\/p>\n LIDAR, which uses low-powered laser beams to detect obstacles, is a common sensor on experimental self-driving cars. But, Coffman noted, because it\u2019s actively emitting laser energy, enemies can easily detect it.<\/p>\n So the robots in the Project Convergence experiment, called \u201cOrigin,\u201d relied on passive sensors: cameras. That meant their machine vision algorithms had to be good enough to interpret the visual imagery and deduce the relative locations of potential obstacles, without<\/em> being able to rely on LIDAR or radar to measure distance and direction precisely. That may seem simple enough to humans, whose eyes and brain benefit from a few hundred million years of evolution, but it\u2019s a radical feat for robots, which still struggle to distinguish, say, a shallow puddle from a dangerously deep pit<\/a>.<\/p>\n \u201cJust with machine vision, they were able to move from Point A to Point B,\u201d Coffman said. But the Army doesn\u2019t just want robots that can find their way around<\/a>: It wants them to scout for threats and targets \u2013 without a human having to constantly stare at the sensor feed.<\/p>\n That\u2019s where Aided Target Recognition comes in. (ATR also stands for Automated<\/em> Target Recognition, but the Army doesn\u2019t like the implication that the software would replace human judgment, so it consistently uses Aided <\/em>instead).<\/p>\n Recognizing targets is another big challenge. Sure, artificial intelligence has gotten scarily good at identifying individual faces in photos posted on social media. But the private sector hasn\u2019t invested nearly as much in, say, telling the difference between an American M1 Abrams tank and a Russian-made T-72, or between an innocent Toyota pickup and the same truck upgunned as a guerrilla \u201ctechnical\u201d with a heavy machinegun in the back. And the Army needs to be able to tell enemy from friendly from civilian in messy real-world combat zones \u2013 and not only from clear overhead surveillance shots, but from the ground, against troops trained to use camouflage and cover to break up easily recognizable silhouettes.<\/p>\n \u201cTraining algorithms to identify vehicles by type, it\u2019s a huge undertaking<\/a>,\u201d Coffman told me. \u201cWe\u2019ve collected and labeled over 3.5 million images\u201d so far to use for training machine-learning algorithms, he said \u2013 and that labeling requires trained human analysts to look at each picture and tell the computer what it was: \u201cThat\u2019s someone sitting there and going, \u2018that\u2019s a T-72; that\u2019s a BMP,\u2019\u201d etcetera ad nauseam, he said.<\/p>\n But each individual robot or drone doesn\u2019t need to carry those millions of images in its own onboard memory: It just needs the \u201cclassifier\u201d algorithms that resulting from running through images through machine-learning systems. Because those algorithms themselves don\u2019t take up a ton of memory, it\u2019s possible to run them on a computer that fits easily on the individual bot.<\/p>\n \u201cWe\u2019ve proven we can do that with a tethered or untethered UAV. We\u2019ve proven we can do that with a robot. We\u2019ve proven we can do that on a vehicle,\u201d Coffman said. \u201cWe can identify the enemy by type and location.\u201d<\/p>\n \u201cThat\u2019s all happening on the edge<\/a>,\u201d he emphasized. \u201cThis isn\u2019t having to go back to some mainframe [to] get processed.\u201d<\/p>\n In other words, the individual robot doesn\u2019t have to constantly transmit real-time, high-res video of everything it sees to some distant human analyst or AI master brain. Sending that much data back and forth is too big a strain on low-bandwidth tactical networks<\/a>, which are often disrupted by terrain, technical glitches, and enemy jamming. Instead, the robot can identify the potential target itself, with its onboard AI, and just transmit the essential bits \u2013 things like the type of vehicles spotted, their numbers and location, and what they\u2019re doing.<\/p>\n \u201cYou want to reduce the amount of information that you pass on the network to a tweet, as small as possible, so you\u2019re not clogging the pipes,\u201d Coffman told me.<\/p>\n But before the decision is made to open fire, he emphasized, a human being has to look at the sensor feed long enough to confirm the target and give the order to engage.<\/p>\n \u201cThere\u2019s always a human that is looking at the sensor image,\u201d Coffman said. \u201cThen the human decides, \u2018yes, I want to prosecute that target.\u2019\u201d<\/p>\n \u201cCould that be done automatically, without a human in the loop<\/a>?\u201d he said. \u201cYeah, I think it\u2019s technologically feasible<\/em> to do that. But the United States Army [is] an ethics-based organization<\/a>. There will be a human in the loop.\u201d<\/p>\n![\"\"](\"https:\/\/www.fie.undef.edu.ar\/ceptm\/wp-content\/uploads\/2020\/10\/Mini-UAV-on-back-200820-A-WL997-1002-768x512-1.jpg\")
![\"\"](\"https:\/\/www.fie.undef.edu.ar\/ceptm\/wp-content\/uploads\/2020\/10\/Coffman-NGCV-CFT-@-AUSA-150x150-1.jpg\")
![\"\"](\"https:\/\/www.fie.undef.edu.ar\/ceptm\/wp-content\/uploads\/2020\/10\/ATAK-feed-from-robot-200824-A-WL997-1061-1024x683-1.jpg\")
![\"\"](\"https:\/\/www.fie.undef.edu.ar\/ceptm\/wp-content\/uploads\/2020\/10\/Robot-from-above-200825-A-WL997-231-1024x559-1.jpg\")