Agentes de voz de inteligencia artificial con diversas capacidades ahora pueden guiar interrogatorios. Este avance ha influido en el dise\u00f1o y las pruebas de agentes de inteligencia artificial del ej\u00e9rcito estadounidense, dise\u00f1ados para interrogar al personal que busca acceder a material clasificado. La situaci\u00f3n surge a medida que crece la preocupaci\u00f3n de que la laxitud de las regulaciones permita a los programadores de IA eludir la responsabilidad por la perpetraci\u00f3n de abuso emocional o cibertortura sin marca por parte de un actor algor\u00edtmico . Cabe destacar que un adolescente presuntamente se suicid\u00f3 , y varios otros sufrieron angustia mental, tras conversar con robots de voz y chatbots autodidactas que utilizaban un lenguaje hostil.<\/p>\n
Artificial intelligence voice agents with various capabilities can now guide interrogations worldwide, Pentagon officials told Defense News. This advance has influenced the design and testing of U.S. military AI agents intended for questioning personnel seeking access to classified material.<\/p>\n
The situation arrives as concerns grow that lax regulations are allowing AI programmers to dodge responsibility for an algorithmic actor\u2019s perpetration of\u00a0emotional abuse<\/a>\u00a0or\u00a0\u201cno-marks\u201d cybertorture<\/a>. Notably, a teenager allegedly\u00a0died by suicide<\/a>\u00a0\u2014 and several others endured mental distress \u2014 after conversing with self-learning voicebot and chatbot\u00a0\u201ccompanions\u201d<\/a>\u00a0that\u00a0dispensed antagonizing language<\/a>.<\/p>\n Now,\u00a0seven years after writing about physical torture<\/a>\u00a0in \u201cThe Rise of A.I. Interrogation in the Dawn of Autonomous Robots and the Need for an Additional Protocol to the U.N. Convention Against Torture,\u201d privacy attorney Amanda McAllister Novak sees an even greater need for bans and criminal repercussions.<\/p>\n In the wake of the use of \u201cchatbots, voicebots, and other AI-enabled technologies for psychologically-abusive purposes \u2014 we have to think about if it\u2019s worth having AI-questioning systems in the military context,\u201d said Novak, who also serves on the World Without Genocide board of directors. Novak spoke to Defense News on her own behalf.<\/p>\n Royal Reff, a spokesperson for the U.S. Defense Counterintelligence and Security Agency (DCSA), which screens individuals holding or applying for security clearances, said in a statement that the agency\u2019s AI voice agent was \u201cexpressly developed\u201d to \u201cmitigat[e] gender and cultural biases\u201d that human interviewers may harbor.<\/p>\n Also, \u201cbecause systems like the interviewer software\u201d are \u201ccurrently being used worldwide, including within the U.S. by state and local law enforcement, we believe that it is important for us to understand the capabilities of such systems,\u201d he stated.<\/p>\n DCSA spokesperson Cindy McGovern added that the development of this technology continues because of its \u201cpotential for enhancing U.S. national security, regardless of other foreign applications.\u201d<\/p>\n After hearing the rationale for the project, Novak registered concerns about the global race for dominance in military AI capabilities.<\/p>\n Odds are high that \u201cno matter how much the government cares about proper training, oversight and guardrails,\u201d cybercriminal organizations or foreign-sponsored cybercriminals may hack into and weaponize military-owned AI to psychologically hurt officers or civilians, she said.<\/p>\n \u201cOnline criminal spaces are thriving\u201d and \u201charming the most vulnerable people,\u201d Novak added.<\/p>\n Investors are betting\u00a0$500 billion that data centers for running AI applications<\/a>\u00a0will ultimately secure world leadership in AI and cost savings across the public and private sectors. The $13 billion\u00a0conversational AI market<\/a>\u00a0alone will nearly quadruple to $50 billion by 2030, as the\u00a0voice generator industry<\/a>\u00a0soars from $3 billion to an expected $40 billion by 2032. Meanwhile, the U.S. Central Intelligence Agency\u00a0has been toying with AI interrogators<\/a>\u00a0since at least the early 1980s.<\/p>\n DCSA officials publicly wrote in 2022 that\u00a0whether a security interview can be fully automated remains an open question<\/a>. Preliminary results from mock questioning sessions \u201care encouraging,\u201d officials noted, underscoring benefits such as \u201clonger, more naturalistic types of interview formats.\u201d<\/p>\n Reff stated that the agency \u201cwould not employ any such system that was not first sufficiently studied to understand its potential,\u201d including \u201cnegative consequences.\u201d<\/p>\n Researchers let volunteers raise concerns after each test, and none have indicated that the voicebot interviewer harmed them psychologically or otherwise, he added.<\/p>\n However,\u00a0experiments<\/a>\u00a0with a recent prototype barred volunteers with mental disorders from participating, whereas actual security interviews typically do not exempt individuals with mental health conditions from questioning, noted Ren\u00e9e Cummings, a data science professor and criminal psychologist at the University of Virginia.<\/p>\n Cummings said that research and testing can never account for every psychological variable that may influence the emotional or cognitive state of the interviewee.<\/p>\n In laboratory or field-testing scenarios, \u201cyou can\u2019t expose someone to extreme torture by an avatar or by a bot to see the result,\u201d said Cummings, who also sits on the advisory council of the AI & Equality Initiative at the Carnegie Council for Ethics in International Affairs. \u201cWe need to focus on gaining a more sophisticated understanding about the impacts of algorithms on the psychological state, our emotions and our brain \u2014 because of AI\u2019s ability to impact in real-time.\u201d<\/p>\n Screening the voicebot screeners<\/b><\/p>\n An early 1980s\u00a0CIA transcript of a purported conversation<\/a>\u00a0between an AI program and an alleged spy highlights the national security world\u2019s long-held interest in automating human-source intelligence, or HUMINT, collection.<\/p>\n Developers designed the chatbot\u2019s algorithms to analyze correlations between suspected spook \u201cJoe Hardesty\u2019s\u201d behaviors and words or phrases in Hardesty\u2019s responses to open-ended questions.<\/p>\n From this analysis, the chatbot pegged some of the suspect\u2019s vulnerabilities, including topics that may hit a raw nerve:<\/p>\n AI: What do you tend to think is my interest in your government?<\/i><\/p>\n Hardesty: I think you would do in my government if you had a chance.<\/i><\/p>\n AI: You are prone to think I would do in your government because?<\/i><\/p>\n Hardesty: You are anti-democratic.<\/i><\/p>\n In 1983<\/a>, the CIA quipped that Joe Hardesty is fortunate that \u201cshould the probing get too discomforting, he will have an option that will not be available to him in a true overseas interview situation \u2014 he can stop the questions with a flick of the \u2018off\u2019 switch.\u201d<\/p>\n Today, Bradley Moss, a lawyer who represents whistleblowers accused of leaking, is not laughing at DCSA\u2019s vision of fully-automated interviews.<\/p>\n \u201cThere would have to be someone observing\u201d the avatar \u201cthat can stop the process, that can come into the room and insert a human element to it,\u201d he cautioned.<\/p>\n Then again, other critics note that a human may not want to intervene. A situation with \u201ca human overseer monitoring the chatbot is fine, unless the human also believes that verbal torture is a good thing. And that is a real possibility,\u201d said Herb Lin, a senior research scholar for cyber policy and security at Stanford University who once held several security clearances.<\/p>\n Despite a\u00a0global ban on torture<\/a>\u00a0and evidence that\u00a0rapport-building interviews are more efficient than ones gunning for confessions<\/a>, intelligence officials continue subjecting interviewees to inhumane treatment.<\/p>\n Novak, the data privacy attorney and anti-genocide activist, said the idea that excessive force gets the job done may encourage some government contractors to set performance metrics for voice algorithms that permit bullying.<\/p>\n \u201cWhile AI has a tremendous ability to bring out the best in human potential, it also has the ability to exacerbate the worst of human potential and to elevate that hostility,\u201d she added.<\/p>\n Parents contend that voicebot and chatbot companions, trained on\u00a0content from online forums<\/a>, show just that.<\/p>\n Character AI<\/a>, the firm that offers virtual relationships with bots patterned after icons and influencers, profits from users glued to their phones. Each user, on average,\u00a0talks two hours a day<\/a>\u00a0with various virtual friends. Character AI officials have\u00a0stated<\/a>\u00a0that,\u00a0after learning of concerns<\/a>, the company reduced the likelihood of users encountering suggestive content and now warns them of hour-long conversations.<\/p>\n Novak commented that the accusations against Character AI illustrate the ability of self-learning voicebots to amplify indecency in training data and to distance trainers from accountability for that offensiveness.<\/p>\n In the national security realm, developers of military AI voice systems that may potentially abuse or terrorize operate in a virtually lawless no-man\u2019s land. AI ethics policies in the world\u2019s most powerful country offer nearly no guidance on de-escalating abusive conversations.<\/p>\n Early U.S.\u00a0responsible<\/a>\u00a0and\u00a0ethical AI policies<\/a>\u00a0in the\u00a0national security space<\/a>\u00a0focused on accuracy, privacy and minimizing racial and ethnic biases \u2014 without regard for psychological impact. That policy gap widened after President Donald Trump\u00a0revoked many regulations<\/a>\u00a0in an\u00a0executive order<\/a>\u00a0aimed at accelerating\u00a0private sector innovation<\/a>.<\/p>\n In general, psychological suffering is often\u00a0\u201cbrushed away\u201d<\/a>\u00a0as a mere allegation since \u201cit leaves no obvious physical scars.\u201d<\/a><\/p>\n The United Nations initially did not address \u201cpsychological torture\u201d under the 1984 Convention Against Torture. Not until five years ago did the term take on significance.<\/p>\n A\u00a02020 UN report now defines \u201ctorture,\u201d in part,<\/a>\u00a0as any technique intended or designed to purposefully inflict severe mental pain or suffering\u00a0\u201cwithout using the conduit\u201d of \u201csevere physical pain or suffering.\u201d<\/a><\/p>\n At bottom, psychological torture targets basic human needs. The interrogation techniques arouse fear; breach privacy or sexual integrity to kindle shame or suicidal ideation; alter sound to heighten disorientation and exhaustion; foster and then betray sympathy to isolate; and instill helplessness.<\/p>\n When an algorithmic audiobot is the conduit of cruelty, neither the UN treaty nor other international laws are equipped to hold humans accountable, Novak said. Without binding rules on the use of AI in the military context, \u201cbad actors will deny any such torture was \u2018intentional\u2019\u201d \u2014 a required element for criminal responsibility that the law created before AI matured, she explained.<\/p>\n Looking to the future, Nils Melzer, author of the 2020 UN report and a former UN special rapporteur on torture, warned that the interpretation of psychological torture must evolve in sync with emerging technologies, \u201csuch as artificial intelligence,\u201d because cyber environments provide\u00a0\u201cvirtually guaranteed anonymity and almost complete impunity\u201d<\/a>\u00a0to offenders.<\/p>\n