We Vetted 2000 AI Skills Before They Reached Developers — Lucas Palma, Nubank
Source: YouTube · AI Engineer · published Jul 29, 2026 · 16:08
Nubank's product security team developed "Skill Vector," a hybrid security review system that vetted over 2,000 AI skills before marketplace distribution, identifying 1,500+ risks and remediating roughly 1,000 of them 0:16 10:00.
Key Takeaways:
• AI skills appear as configuration but behave like supply chain dependencies, expanding the traditional supply chain to include skills, plugins, MCP servers, agent rules, and hooks 1:18 2:12
• Dangerous behaviors include hardcoded tokens causing data leaks, destructive shell commands, and excessive permissions—even typos carry risk in regulated financial environments 4:38
• Skill Vector uses a hybrid approach: deterministic regex checks for clear patterns plus LLM review for contextual analysis, with findings reported as PR comments and exported as SARIF 6:02 7:02
• The system scans for unsafe instructions, behavior drift, destructive commands, improper credential handling, over-broad permissions, and risky MCP usage, blocking or requiring remediation based on severity 9:18
• A key lesson was detecting "prompt-level confirmation" where the AI asks itself for approval, creating a false sense of human-in-the-loop oversight 12:56
Security teams must protect the entire AI-assisted developer workflow, not just generated code—a principle now extending to MCP servers, rules, and third-party plugins.
Sources:
- 0:16 Introduction and scale of AI skill vetting
- [1:18](https://www.youtube.com/watc
Generate CPE Credits
Generate a professional CPE document from this video's transcript.
Estimated credit: 0.5 CPE hours
Estimate uses the video runtime (1 hour ≈ 1 CPE, rounded to the nearest 0.5, minimum 0.5, maximum 2.0). The final amount can be lower after review, never higher.
Topic: Cybersecurity. Commonly maps to: Security and Risk Management, Security Operations. Exact CISSP domains are assigned during generation.
Free account. One generation at a time, with a daily limit.
CPEBuddy is independent and not affiliated with or endorsed by ISC2, ISACA, or any certification body. Exports are formatted for common CPE submissions; acceptance is at your certification body's discretion.
Transcript Preview
First 800 characters of the transcript
[music] Hello everyone. Good afternoon. Today I'm going to talk about how we vetted 2,000 AI skills before they reach a developers. But before I before of that, I'm Lucas Palma, but many people call me LP. I'm the product security manager at New Bank, the product security structures, uh structure that's within security, looking upon how we make code safe and supporting engineers, product managers and everybody to making our products safer. I have uh over a decade of experience in financial services engineering background also a lot of years working here at security and a close relationship with the part that I love which is innovation. So before beginning I believe I want to bring to you uh why are we here. So one thing that's important for all of you to understand that the now that we are…