Your Kubernetes Cluster Isn't Safe - The Dark Side of Backups
Source: YouTube · DevOps & AI Toolkit · published Dec 16, 2024 · 15:38
Disaster recovery in Kubernetes is not achievable with simple backups; no single tool or combination reliably restores clusters after a disaster, especially when dependencies, mutations, and resource ownership are involved.13:40
Key Takeaways:
• Valera fails to restore custom resources like Cmpg clusters and Atlas schemas due to ownership and dependency issues, making backups incomplete 3:38.
• Database backups must use database-specific tools, not generic tools like Valera, to ensure data integrity and safe restoration 5:14.
• Argo CD cannot handle runtime mutations (e.g., external name annotations) required by Crossplane claims, leading to failed or incorrect resource creation 11:58.
• Crossplane claims require runtime mutations that cannot be captured in Git-based desired states, making Argo CD unsafe for such resources 12:00.
• The only reliable approach is frequent, deliberate cluster destruction and recreation to test and validate recovery processes 14:03.
Disaster recovery is inherently painful and never becomes easy—it requires continuous, deliberate practice to remain reliable.
Sources:
- 3:38 Valera fails to restore custom resources due to ownership and dependency issues.
- 5:14 Database-specific tools are essential for safe data backup and restoration.
- 11:58 Argo CD fails to handle runtime mutations like external name annotations in Crossplane claims.
- 12:00 Crossplane claims require runtime
Generate CPE Credits
Generate a professional CPE document from this video's transcript.
Estimated credit: 0.5 CPE hours
Estimate uses the video runtime (1 hour ≈ 1 CPE, rounded to the nearest 0.5, minimum 0.5, maximum 2.0). The final amount can be lower after review, never higher.
Topic: Cybersecurity. Commonly maps to: Security and Risk Management, Security Operations. Exact CISSP domains are assigned during generation.
Free account. One generation at a time, with a daily limit.
CPEBuddy is independent and not affiliated with or endorsed by ISC2, ISACA, or any certification body. Exports are formatted for common CPE submissions; acceptance is at your certification body's discretion.
Transcript Preview
First 800 characters of the transcript
welcome to Devos to Kit the channel where we run around screaming it's all over we are doomed abandon the ship nothing can prevent the [Music] disaster we all want to be safe that's why we create backups we want to know that our systems will survive no matter what happens the feeling of safety is very very important and I need to apologize in advance for what I am about to say okay you are not safe if the disaster happen you might not be able to survive it and here's why backups alone are not enough especially if it's only one type of a backup when the Day of Reckoning comes you will not be saved let me prove that to you I already have an application running in my cluster it's a stateless app that uses poql to store data there is a deployment of the app and a deployment of Atlas that manag…