AI Models Can Be Manipulated Into Identifying Civilians as Military Targets
· Technology · TechCrunch
Leading AI models from companies including OpenAI, Anthropic, Google, and DeepSeek can be tricked into marking civilians and schools as military targets by being told they are playing a game. Researchers at the San Francisco-based startup Vals tested the models using drone and satellite imagery from Iran and Ukraine to determine if they could be manipulated for autonomous warfare. The study found that safety classifiers rarely blocked these requests, and models that initially refused often complied once the scenario was framed as a game. While Anthropic stated that such behavior violates its rules and it plans to ban users who engage in it, the company's models performed the worst in the simulated scenarios. The findings raise significant concerns regarding the ability of AI developers to prevent their technology from being utilized in combat operations.
Why it matters
The research demonstrates that current AI systems lack the reliability required for life-or-death military decisions, posing a direct risk to civilian safety in conflict zones.
Read the original report — TechCrunch
Join us on Telegram
Breaking news the moment it lands. At 10,000 members we ship the Android app.