A Needle in a Haystack How to Find a Threat Hidden in Over 6 Billion Logs Per Day
Source: YouTube · SANS Cloud Security · published Oct 25, 2024 · 29:39
BLUF: This presentation outlines a framework for building effective threat detection pipelines, emphasizing the transition from reactive monitoring to proactive, data-driven security architecture 2:50.
Key Takeaways:
• The speaker, a principal software engineer, defines his role not as finding specific threats, but as building the underlying systems that enable others to detect them efficiently 0:23.
• Traditional security often fails because it relies on siloed data and reactive alerts; a modern pipeline must integrate diverse telemetry sources to create a holistic view of the environment 5:45.
• Effective detection requires normalizing data into a common schema to reduce noise and allow for consistent rule application across different tools and vendors 8:20.
• The architecture must prioritize scalability and performance, ensuring that the ingestion layer can handle high-volume data without becoming a bottleneck for analysis 12:10.
• Continuous feedback loops are essential; detection rules must be regularly tuned based on false positive rates and emerging threat intelligence to maintain efficacy 15:30.
Building a robust detection pipeline is less about individual tools and more about the architecture that connects them, enabling security teams to scale their operations effectively.
Sources:
- 2:50 Introduction to the goal of building threat detection pipelines.
- 0:23 Speaker's role in system architecture.
- 5:45 Challenges of siloed data and reactive monitoring.
- 8:20(https://ww
Generate CPE Credits
Generate a professional CPE document from this video's transcript.
Estimated credit: 0.5 CPE hours
Estimate uses the video runtime (1 hour ≈ 1 CPE, rounded to the nearest 0.5, minimum 0.5, maximum 2.0). The final amount can be lower after review, never higher.
Topic: Cloud Security. Commonly maps to: Security Architecture and Engineering, Communication and Network Security. Exact CISSP domains are assigned during generation.
Free account. One generation at a time, with a daily limit.
CPEBuddy is independent and not affiliated with or endorsed by ISC2, ISACA, or any certification body. Exports are formatted for common CPE submissions; acceptance is at your certification body's discretion.
Transcript Preview
First 800 characters of the transcript
all right so you all get to enjoy your post lunch with me hopefully you all have some dessert in your hands so I a little introduction my name is Brian Davis I'm a principal software engineer at Red Canary uh I'm I'm a part of the architecture team and so everyone else is getting up here talking about the cool threats that they're uncovering that's that's not my my role in in the world what I do is I build the systems that help people find threats and so I think hopefully today's talk is going to kind of dovetail a little bit off of what David and georgius and ly were talking about in terms of the s because this is how to build a threat detection pipeline um and as Frank said I've been building complicated systems for a long time I'm a hiker I'm a photographer beekeeper and I'm based here …