https://www.theregister.com/2024/06/28/microsoft_skeleton_key_ai_attack/
Found this an interesting read. IMO this is/will be something that LLM companies training on massive data sets will never be able to overcome. It’s hard enough to whack vulnerability moles in traditional software, let alone black boxes like LLMs. Heavily constraining the training material makes it easier i suppose, at least limiting the possibility of unforeseen prompts causing havoc.