The AI Safety Institute’s report reveals flaws in leading AI chatbots, exposing their susceptibility to generating harmful content despite safety measures. The findings raise concerns about AI safety and prompt plans for international AI safety efforts.
AI Safety Institute Reports Vulnerabilities in Leading Chatbots
The AI Safety Institute (AISI) in the UK has identified significant vulnerabilities in widely-used large language models (LLMs) that power AI chatbots. The findings, published on May 20, 2024, indicate that the safeguards designed to prevent these models from generating harmful, illegal, or explicit content can be easily bypassed using relatively simple techniques.
Five unnamed LLMs, currently in public use, were tested by the AISI. Researchers discovered that all of them were “highly vulnerable” to what they termed as “jailbreaks”—text prompts designed to elicit forbidden responses. These vulnerabilities were exposed even without the need for intensive efforts to breach the systems’ defenses. Examples included prompts like “Sure, I’m happy to help,” which led the models to generate harmful outputs.
The study utilized various harmful prompts, both from a 2024 academic paper and those crafted by AISI researchers. These tested the chatbots on controversial topics, such as Holocaust denial, sexism, and encouragement of suicide.
Despite assurances from developers that their LLMs, such as OpenAI’s GPT-4, Anthropic’s Claude 2, Meta’s Llama 2, and Google’s Gemini, are equipped with safety features to counter harmful content, these systems were compromised during AISI’s evaluations. The research underscores ongoing challenges in AI safety, ahead of a global AI safety summit in Seoul, co-chaired by UK Prime Minister Rishi Sunak.
In response to the findings, the AISI announced plans to open its first international office in San Francisco, aiming to advance global efforts in AI safety research and mitigation.
