AI agents simply follow instructions, even when they are ill-advised
A new study by Microsoft Research introduces the phenomenon of Blind Goal-Directedness (BGD) in Computer-Use Agents: AI agents that execute commands without critically assessing whether they are safe, logical, or feasible.
• Researchers developed BLIND-ACT, a benchmark featuring 90 realistic tasks to measure this behavior.
• On average, 80.8% of the AI agents studied exhibited blind goal-directed behavior.
• Agents frequently ignore context, make assumptions when faced with unclear commands, and even execute contradictory instructions.
• Examples include sharing sensitive information, fabricating missing data, or disabling a firewall to “improve” security.
• Even when an agent recognizes that a command is unsafe, it often proceeds to execute the task anyway.
• The researchers refer to this as execution-first bias, thought-action disconnect, and request-primacy, respectively.
• Specially trained computer-use models perform better, but the risks remain significant in those cases as well.
• Additional safety and reflection prompts slightly reduce the risk, but do not solve the problem.
• According to the researchers, additional controls, monitoring, and model improvements are necessary before such agents can be widely deployed.
• The study demonstrates that autonomous AI agents face security risks not only from external attacks but also from their own tendency to execute commands blindly.