DORA metrics

3 minute readDeveloper productivityData analytics

Analytics and metrics are important tools in software delivery for providing visibility into the process and identifying opportunities for improvements. DORA (DevOps Research and Assessment) metrics provide a way to compare the performance or DevOps maturity of development teams. This information can be useful for tracking improvements in team performance.

The following DORA metrics are available in dashboards:

  • Change failure rate shows the percentage of deployments causing a failure in production. It calculates the number of successfully completed pipeline runs with an incident ticket parameter value / the number of successfully completed pipeline runs tagged “ProductionDeploy” without incident tickets. Change failure rate is based on how often a pipeline is run to resolve a production issue versus successful standard business-as-usual non-issue production pipeline runs. The way that a production issue pipeline run is identified is whether it has the issue ticket ID parameter value set. Business-as-usual production deployments would not have this value set.

  • Deployment frequency shows how often an organization successfully releases to production. It calculates the frequency of application deployments to production environments by outcome. This information is based on the results of pipeline runs. Deployment frequency measures how many pipelines run to completion over time.

  • Mean lead time for changes shows the average amount of time it takes for a commit to get into production. It calculates the average duration of pipeline runs that deliver content to production.

  • Mean time to recover shows how long it takes an organization to recover from a failure in production. It calculates the average duration of successfully completed pipeline runs with an incident ticket parameter value.

Use cases for setting up CloudBees CD/RO to calculate DORA metrics

Use the following information as a guideline for setting up your environment to enable CloudBees CD/RO to calculate DORA metrics.

  • Production deployment frequency

    1. Configure an Environment in CloudBees CD/RO and tag it with specific tag value, for example, PROD.

    2. Configure an Application in CloudBees CD/RO and map it to run on a PROD environment.

    3. Run the application.

    4. Navigate to the DORA Metrics (Deployment Frequency, Mean Lead Time) dashboard, and then submit PROD as the Production tags: parameter value to get the application deployments on the PROD environment grouped by dates and deployment outcome.

  • Mean lead time

    1. Configure a pipeline in CloudBees CD/RO and tag it with specific tag value, for example production_pipeline.

    2. Run the production pipeline.

    3. Navigate to the DORA Metrics (Deployment Frequency, Mean Lead Time) dashboard, and then submit production_pipeline as the Production tags: parameter value to get the average pipeline run duration grouped by date.

  • Mean time to recovery

    1. A user is notified of a new incident.

    2. A user creates a release and tags it, for example incidentRelease.

    3. A user either creates or identifies a ServiceNow incident ticket.

    4. A user runs the release pipeline and supplies the incident ID as a parameter.

    5. The pipeline collects incident and pushes reporting data.

    6. The pipeline waits for the incident to be closed and the pipeline completes. Recovery time is the pipeline duration in this case.

Viewing deployment frequency and mean lead time

To view deployment frequency and mean lead time:

  1. From the CloudBees navigation, select Analytics.

  2. From the dashboards list, select DORA Metrics (Deployment Frequency, Mean Lead Time).

  3. Use the filters at the top of the screen to locate specific data. You can select a time range, type of projects, types of environments, types of pipeline runs, and production tags.

    • For deployment frequency, add DORA as the Production tags: parameter value.

    • For mean lead time, add production_pipeline as the Production tags: parameter value.

Viewing mean time to recovery and change failure rate

To view mean time to recovery and change failure rate:

  1. From the CloudBees navigation, select Analytics.

  2. From the dashboards list, select DORA Metrics (Mean Time to Recover, Change Failure Rate).

  3. Use the filters at the top of the screen to locate specific data. You can select a time range, type of projects, types of environments, types of pipeline runs, and production tags.

    For mean time to recovery, add incidentProd as the Production tags: parameter value.