Summary: use digest as the way to refer to docker images in kubernetes resources, put commit id in image labels. An idempotency and immutable infrastructure has a slew of benefits. I am a firm believer of it and I did my best to keep several projects in that way. In the past few weeks, I helped one friend to resolve system structure, performance issue and development experience of one of his projects.