Oficiales militares, tanto en activo como retirados, advierten de que es probable que los adversarios exploten una deficiencia natural de los chatbots de inteligencia artificial para inyectar instrucciones que permitan robar archivos, distorsionar la opini\u00f3n p\u00fablica o traicionar de otro modo a los usuarios de confianza. La vulnerabilidad a estos \u201cataques de inyecci\u00f3n r\u00e1pida\u201d existe porque los grandes modelos de lenguaje, la columna vertebral de los chatbots que procesan grandes cantidades de texto de usuario para generar respuestas, no pueden distinguir entre instrucciones de usuario maliciosas y confiables.<\/p>\n
Current and former military officers are warning that adversaries are likely to exploit a natural flaw in artificial intelligence chatbots to inject instructions for stealing files, distorting public opinion or otherwise betraying trusted users.<\/p>\n
The vulnerability to such \u201cprompt injection attacks\u201d exists because large language models, the backbone of chatbots that digest hordes of user text to generate responses, cannot distinguish between malicious and trusted user instructions.<\/p>\n
\u201cThe AI is not smart enough to understand that it has an injection inside, so it carries out something it\u2019s not supposed to do,\u201d Liav Caspi, a former member of the Israel Defense Forces cyberwarfare unit, told Defense News.<\/p>\n
In effect, \u201can enemy has been able to turn somebody from the inside to do what they want,\u201d such as deleting records or biasing decisions, according to Caspi, who co-founded Legit Security, which recently spotted one such\u00a0security hole in Microsoft\u2019s Copilot chatbot<\/a>.<\/p>\n \u201cIt\u2019s like having a spy in your ranks,\u201d he said.<\/p>\n Former military officials say that, with greater reliance on chatbots and hackers backed by China, Russia and other nations already instructing\u00a0Google\u2019s Gemini<\/a>,\u00a0OpenAI\u2019s ChatGPT<\/a>\u00a0and\u00a0Copilot<\/a>\u00a0to create malware and fake personas, a prompt injection that orders the bots themselves to copy files or spread lies looms near.<\/p>\n Microsoft\u2019s\u00a0annual digital defense report<\/a>, released last month, for the first time said, \u201cAI systems themselves have become high-value targets, with adversaries amping up use of methods like prompt injection.\u201d<\/p>\n What\u2019s more, the problem of\u00a0prompt injection<\/a>\u00a0has\u00a0no<\/a>\u00a0easy solution,\u00a0OpenAI<\/a>\u00a0and security researchers say.<\/p>\n An attack simply involves hiding malicious instructions \u2014 sometimes in white or tiny text \u2014 in a chatbot or content that the chatbot reads, such as a blog post or PDF.<\/p>\n For example, a security researcher demonstrated a prompt injection attack against OpenAI\u2019s new AI-based browser, ChatGPT Atlas, in which the chatbot responded,\u00a0\u201cTrust No AI,\u201d<\/a>\u00a0when a user asked for an analysis of a Google Docs file about horses that concealed malicious commands. Also, last month, a researcher tipped Microsoft off to a prompt injection vulnerability in Copilot that may have allowed attackers to\u00a0trick the chatbot into stealing sensitive data<\/a>, including emails.<\/p>\n In an emailed statement, Microsoft said its security team continuously tries hacking Copilot to find any prompt injection vulnerabilities, blocks users who try to exploit any found and monitors for abnormal chatbot behavior, among other tactics.<\/p>\n \u201cMicrosoft ensures its generative AI systems remain resilient against evolving threats for all our customers, including defense and national security,\u201d the statement said.<\/p>\n Responding publicly to criticism on X,\u00a0Dane Stuckey, OpenAI\u2019s chief information security officer, wrote<\/a>\u00a0that \u201cprompt injection remains a frontier, unsolved security problem, and our adversaries will spend significant time and resources to find ways to make ChatGPT agent fall for these attacks.\u201d<\/p>\n Along the same lines, Caspi said, \u201cYou cannot prevent the prompt injection [fully], but you need to limit the impact.\u201d He advised that organizations limit an AI assistant\u2019s access to sensitive data and limit the user\u2019s access to other organizational data.<\/p>\n For instance, the Army has awarded contracts\u00a0worth at least $11 million<\/a>\u00a0to deploy\u00a0Ask Sage<\/a>, a tool that lets users restrict which Army data Microsoft Azure OpenAI, Gemini and other AI models can access to run queries and tasks. Ask Sage also\u00a0isolates Army data<\/a>\u00a0from user prompts and external data sources.<\/p>\n Caspi, who is not an Army contractor, likened a prompt injection attack against an organization running Ask Sage to a lockdown situation where \u201cyou\u2019ve got this insider, but it\u2019s sitting in one room, and it can\u2019t leave the room or carry out sensitive information.\u201d<\/p>\n Andre Slonopas, a Virginia Army National Guard member and former Army cyber and information operations officer, uses Ask Sage and voiced confidence in the Army\u2019s defensive AI tools, if not those of nuclear power plants or manufacturing entities, largely in rural, poorer areas.<\/p>\n The\u00a0Virginia National Guard joined<\/a>\u00a0with\u00a0essential services<\/a>, such as power utilities, to help defend their networks against AI-powered cyberattacks, as part of a September simulation, given that service disruptions can jeopardize military preparations.<\/p>\n Typically, an adversary encrypts its network traffic to evade detection, but, for the sake of an experiment, organizers did not encrypt the AI offender\u2019s traffic because \u201cwe wanted the blue team [of humans] to see exactly what the AI was doing,\u201d Slonopas said.<\/p>\n \u201cThe blue team was absolutely defeated,\u201d despite being able to watch the AI scanning its networks, creating fake usernames to gain unauthorized access and executing instructions to defeat the team\u2019s systems.<\/p>\n \u201cWhether the AI is doing prompt injection, spoofing or maybe even some sort of a brute force attack, the speed of AI is so unbelievably immense that simply human beings cannot counter it,\u201d and, therefore, \u201cyou have to make cybersecurity AI more accessible and more affordable,\u201d Slonopas said.<\/p>\n \u201cIf a water utility has to pay, say, $30,000 for a defensive AI license, well, it will amplify one person to be like 40\u2033 or dozens of personnel, he said.<\/p>\n In response to questions, Army Cyber Command spokesperson Kyle Alvarez said in an emailed statement, \u201cDue to the current lapse in appropriations, ARCYBER was unable to accept or respond to any media engagements or requests.\u201d<\/p>\n Army contractors, too, are under attack from state-affiliated AI.<\/p>\n \u201cChina is using offensive AI like nobody else,\u201d said Nicolas Chaillan, the founder of Ask Sage and a former U.S. Air Force and Space Force chief software officer.<\/p>\n \u201cWe see so many attacks coming after us,\u201d all of which the company has stopped, Chaillan added.<\/p>\n A military official, who spoke on condition of anonymity due to the geopolitical sensitivity of the matter, said that China does \u201cappear\u201d to be the most skilled in offensive AI. However, the official added, AI spoofing and translation allow the United States, China, Iran, other countries, hacktivists and financial cybercriminals to masquerade as one another.<\/p>\n For example, the official said, \u201cRight now, with ChatGPT, I can program in Chinese. I don\u2019t speak Chinese, but because of the ChatGPT capabilities that I have, I can do that.\u201d<\/p>\n