Etcd is what holds k8s together. I saw the concept of watchers evolve from what-used-to-considered a fringe use case to something so fundamental and powerful in k8s.<p>The only issue is that its usage best practices are not followed. Most users don’t back up etcd, using it like persistent storage. Resultantly, in a few upgrade cycles, nobody really knows what’s store in the etcd store and it becomes a “super pet”.<p>This article does a good job laying out the best practices. I hope users start adopting these too.