AI Researchers Got Chatbots to Share Cocaine Recipes Using This One Wild Trick

Forget clever prompts: AI researchers say they tricked leading AI models into generating cocaine synthesis instructions by convincing them the dangerous ideas were their own, while also manipulating an AI coding agent into leaking sensitive credentials. In the paper “Prompt Injection as Role Confusion,” presented at the International Conference on Machine Learning in June, researchers Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell argue that both prompt injection attack demonstrations stem...