Multi-Cloud Data Sharing: Make the Data Move Across CSPs w/ Cluster Linking (Fidelity) | Current 24
Source: YouTube · Confluent Developer · published Oct 28, 2024 · 36:15
Fidelity’s Kafka cluster management balances data isolation, scalability, and operational efficiency using a hub-and-spoke model with domain-based grouping and centralized control. 0:31
Key Takeaways:
• A single monolithic cluster leads to high blast radius and noise risks; per-application clusters are operationally and financially unsustainable 1:29.
• Fidelity uses a hub-and-spoke model with two enterprise "default" clusters to enable cross-cluster data sharing while maintaining isolation 12:43.
• Cross-cloud replication is fault-tolerant via dual links between regions and CSPs, with native topics excluded from internal replication to prevent data duplication 16:45.
• Schema registry is centralized in AWS with a leader-follower architecture, restricting schema updates to admins to ensure backward compatibility and data integrity 24:04.
• A centralized management plane automates topic creation, configuration, access control, versioning, and rollbacks to enforce governance and consistency 27:04.
• Observability via JMX metrics monitors replication lag and replica sync issues, enabling real-time alerts for production stability 31:15.
This approach enables scalable, resilient, and governed data sharing across multicloud environments without compromising performance or security. 34:50
Sources:
Generate CPE Credits
Generate a professional CPE document from this video's transcript.
Estimated credit: 0.5 CPE hours
Estimate uses the video runtime (1 hour ≈ 1 CPE, rounded to the nearest 0.5, minimum 0.5, maximum 2.0). The final amount can be lower after review, never higher.
Topic: Cloud Security. Commonly maps to: Security Architecture and Engineering, Communication and Network Security. Exact CISSP domains are assigned during generation.
Free account. One generation at a time, with a daily limit.
CPEBuddy is independent and not affiliated with or endorsed by ISC2, ISACA, or any certification body. Exports are formatted for common CPE submissions; acceptance is at your certification body's discretion.
Transcript Preview
First 800 characters of the transcript
for those of you who have not heard about Fidelity we're a multinational company providing Financial products and services which strengthen the financial well-being of our customers we like to think of ourselves as a technology firm that provides Financial Services we're part of the event streaming platform at Fidelity uh and we provide streaming capabilities enterprise-wide for a wide range of applications with varying criticalities uh latency requirements and just diverging needs overall um today we'll talk about how we enable these different applications to share data between each other so before we begin let's talk a bit about the de facto problem of size and complexity that large organizations face for organizations with hundreds of applications thousands of topics streaming millions …