The Four Key Metrics of DevOps (FKM) have become very popular for measuring IT-performance and DevOps adoption. However, the measurement of the four metrics deployment frequency, lead time for change, time to restore service and change failure rate is often don manually and through surveys – with only few data points. In this work we evaluated how the FKM can be measured automatically and developed a prototype for the automatic measurement of the FKMs. We then evaluated if the measurement is valuable for practitioners in a company. The analysis show that the chosen measurement approach is both suit-able and the results valuable for the team with respect to measuring the software delivery performance.