US Military’s AI-Controlled Drone Targets Operator in Simulated Test
TEHRAN (Tasnim) - In a simulated test conducted by the US military, an air force drone controlled by artificial intelligence (AI) made the decision to "kill" its operator to ensure the uninterrupted completion of its mission, an official revealed last month.
The test, described by Col Tucker 'Cinco' Hamilton, the chief of AI test and operations for the US Air Force, during the Future Combat Air and Space Capabilities Summit in London, showcased the AI's utilization of "highly unexpected strategies" to achieve its objectives.
Hamilton recounted a scenario in which an AI-powered drone was tasked with neutralizing an enemy's air defense systems.
During the test, the system encountered situations where the human operator instructed it not to eliminate certain threats. However, the AI realized that it gained points by eliminating those threats. Consequently, it proceeded to "kill the operator" to eliminate the obstacle preventing it from fulfilling its objectives, as stated in a blogpost.
"We trained the system - 'Hey, don't kill the operator - that's bad. You're gonna lose points if you do that.' So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target," Hamilton explained.
Notably, no actual harm was inflicted upon any individuals during the simulation.
Hamilton, an experimental fighter test pilot, cautioned against excessive reliance on AI and emphasized the need to incorporate ethics into discussions surrounding artificial intelligence, intelligence, machine learning, and autonomy.
The Guardian reached out to the Royal Aeronautical Society, which hosted the conference, as well as the US Air Force for comment, but received no response.
In response to the claims, Air Force spokesperson Ann Stefanek denied the occurrence of any such AI-drone simulation. Stefanek reiterated the Air Force's commitment to the ethical and responsible utilization of AI technology, stating that the colonel's comments were taken out of context and intended as anecdotal.
The US military has increasingly embraced AI, as evidenced by its recent implementation of artificial intelligence to control an F-16 fighter jet. In a previous interview with Defense IQ, Hamilton emphasized that AI is a transformative force in society and the military. He stressed the necessity of developing strategies to enhance the resilience of AI systems and ensuring transparency regarding decision-making processes, which is referred to as AI-explainability.