DEF CON 34 - BugBounty - Finding Prompt Injection Vulns via Mutated Prompt Gen - B Steele, A Garg

[BLUF] Microsoft researchers present the Mutated Prompt Generator (MPG) framework, a divergent search tool that automates the discovery of residual prompt injection vulnerabilities by mutating payloads to test if patches generalize beyond specific reported exploits 2:15-2:30.

Key Takeaways:
• Indirect Prompt Injection Attacks (IPIA) allow attackers to hide malicious instructions in external content (like web pages) that AI agents process, leading to data exfiltration via hidden links or images without user interaction 1:50-2:15.
• Attackers exploit the leniency of markdown and HTML parsers, using malformed syntax, Unicode characters, or reference links to bypass simple regex-based defenses 6:21-8:15.
• Traditional security testing focuses on finding a single successful "convergent" jailbreak, whereas MPG employs a "divergent" search to map the breadth of the attack surface and measure residual exploitability 12:15-13:15.
• The MPG framework classifies token segments into "carrier" (benign) and "injection" (malicious) tokens, applying specific mutation operators to each to generate variant payloads that test the robustness of existing safeguards 16:21-18:15.
• Using a subtree viability scoring algorithm, MPG evaluates the success rate of mutated variants to determine if a vulnerability is broadly generalizable, indicating a need for architectural mitigation rather than targeted patches 22:45-24:45.
• Internal testing showed that while 35% of vulnerabilities were initially successful, the MPG framework resurfaced an additional 19% of cases that were previously deemed patched, highlighting significant residual risk 24:48-25:25.

The MPG framework provides a critical defensive use case for post-incident validation and pre-release gating, helping developers ensure AI features do not reach production with exploitable vulnerabilities.

Sources:

• 2:15-2:30 Introduction to the research goal of measuring residual exploitable surface.
• 1:50-2:15 Explanation of indirect prompt injection attacks and data exfiltration techniques.
• 6:21-8:15 Discussion on how markdown and HTML parser leniency aids attackers.
• 12:15-13:15 Contrast between convergent search (finding one exploit) and divergent search (mapping surface).
• 16:21-18:15 Description of the MPG framework's token classification and mutation strategy.
• 22:45-24:45 Details on the subtree viability scoring algorithm and its role in determining mitigation needs.
• 24:48-25:25 Presentation of test results showing resurfaced vulnerabilities via mutation.