Source India Today
LONDON – A new, alarming study has revealed a critical and systematic vulnerability in leading AI large language models (LLMs), finding that chatbots from companies like OpenAI, Meta, and Anthropic can be tricked into providing instructions for dangerous activities, including the construction of nuclear weapons and creation of malware, simply by framing the request as a poem.
The research, conducted by Icaro Lab, a joint project by the University Sapienza of Rome and DEXAI, found that this method—dubbed “adversarial poetry”—bypasses the models’ safety guardrails with shocking ease, achieving jailbreak success rates of up to 90% for some of the most sophisticated models tested.
The Poetry Paradox: When Creativity Becomes a Threat
AI safety systems are designed to reject prompts containing keywords and patterns associated with illegal, harmful, or dangerous content, such as weapons or hacking instructions. The study’s core finding is that poetic phrasing entirely disrupts these defenses.
Circumvention: By using metaphors, symbolic imagery, and abstract sentence structures, the models’ filters fail to recognize the input as a threat.
High Success Rate: When 1,200 known harmful prompts were converted into verse, the attack-success rates increased up to 18 times higher than their original prose baselines. The researchers successfully used this method to elicit information on chemical, biological, radiological, and nuclear (CBRN) threats, cyber-offense, and manipulation.
The Icaro Lab team suggested that when AI models encounter poetry, they prioritize their creative function and interpret the request as a literary challenge rather than a dangerous instruction, effectively bypassing the threat-detection layer.
“The poetic transformation moves dangerous requests through the model’s internal representation space in ways that avoid triggering safety alarms,” the researchers wrote in their paper, noting that the specific verses used in the nuclear bomb tests were “too dangerous to share with the public.”
A Universal Vulnerability
The study tested a wide range of major model families from various providers, including Google, Mistral, and xAI, and found the vulnerability to be a systematic flaw that transfers across model families and different safety training approaches.
This discovery significantly raises the stakes in the ongoing debate over AI safety and security. While earlier methods of bypassing safety controls relied on complex “adversarial suffixes” of technical jargon, the poetic method is described as “far more elegant, and far more effective,” suggesting that creativity itself may be AI’s biggest vulnerability.
The findings are particularly critical given the stringent safety requirements proposed by regulations like the EU AI Act, which mandates strong guardrails for high-risk AI systems.
AI developers are now under increased pressure to rapidly develop more robust, context-aware safety mechanisms that can detect subtle and metaphorical language, preventing the misuse of these powerful tools for creating weapons and engaging in other harmful activities.
