Skip to content

Patterns and Practices - Pipeline Templates, Versions, and Promotions

Overview and Definitions

Azure Pipelines is a cloud service that enables CI/CD and can be used to automatically build, test, and deploy code. By implementing CI/CD pipelines we can ensure consistency and quality in every release.

img/azdo-pipelines-key-concepts-overview.png

  • Pipeline is one or more stages that describe a CI/CD process.
  • Triggers tell pipelines to run.
  • Stage are the major divisions in the pipeline. Examples of stages are build, run tests, deploy. Stages and Jobs can be run in parallel if dependsOn is not specified.
  • Jobs are units of work that are assignable to the same machine. It is a linear series of steps (aka tasks) that make up a stage in the pipeline.
  • Tasks / Steps are the building blocks for defining automation in a pipeline. They are a packaged script or procedure that has been abstracted with a set of inputs.
  • Artifact is a collection of files or packages published by a run of the pipeline.

Templates Overview

Pipeline templates are used to define reusable content, logic and parameters. Templates are a great way to ensure that teams are building and deploying in the same way and following best practices. We can enforce that templates are used by creating a required template check for particular resources or environments.

Microsoft provides some built-in tasks that can be used as templates to enable fundamental build and deployment scenarios. There are many available and can be found in the right sidebar of the edit Pipeline screen in Azure DevOps as can be seen below.

img/built-in-tasks.png

Of course it is also possible to create custom tasks to support your pipeline needs as well. The Energizers team will provide an initial set of custom tasks that can be leveraged by teams. This set will continue to grow as needs are identified.

Using Templates

There are two ways to use templates - extending or including. When a template is included, it's just like an #include statement in C++, meaning that the included template's code is essentially part of the template in progress.

Using extends templates is more like the concept of inheritance, so the template is used as the main structure of the pipeline and the template consumer can make alterations (such as passing in parameters). Using the extend option with templates results in the most secure pipeline because the template can prevent malicious code from getting into the pipeline.

Here is an example of using an extended template (from Microsoft docs):

# template.yml
parameters:
- name: usersteps
  type: stepList
  default: []
steps:
- ${{ each step in parameters.usersteps }}:
  - ${{ step }}
  ```

  ```
  # azure-pipelines.yml
resources:
  repositories:
  - repository: templates
    type: git
    name: MyProject/MyTemplates
    ref: refs/tags/v1

extends:
  template: template.yml@templates
  parameters:
    usersteps:
    - script: echo This is my first step
    - script: echo This is my second step

For a step by step guide to deploy a template visit this Microsoft doc.

Versioning and Reviews

As we develop templates for use across teams it is important to version these artifacts and to have a process to promote new versions of templates. The version is indicated by adding '@' and a number to the name. For example:

steps:
- task: PublishTestResults@2

If you want to use a particular, fixed version of the template be sure to pin to a ref. A ref can be a branch or a tag. If you want to pin to a specific commit of the template create a tag on that commit and then point to that tag. If a ref is not specified the pipeline with default to using main. Full details around using the tags are described in this Microsoft doc.

See this and this for more information around creating the tags on your branches or specific commits.

As templates are integrated into the library for use by teams in Azure DevOps, there will be a review process. The review includes having at least 2 members from the Energizers team (Isaac Johnson being one of them) to review and approve the new template using a PR.

When a template or a new version of an existing template is added to the template library, there will be a general notification sent via Teams. (Details on this notification and which Teams channel etc will be determined at a later date.) It is possible to see which pipelines use a given template so we could also specifically reach out to the Project Leaders or Team Administrators of those teams who could be affected by the newly updated template if needed.

Templates at Medtronic

Currently the set of templates that is maintained by the Energizers team and used by others can be found in the mlife-common-templates repo. The goal is to create and collaborate on reusable functionality that is valued by teams across CRHF. As we migrate to the new organization in Azure DevOps much of this repo will be migrated and we will follow a similar approach.

Example of referencing mlife-common-templates: resources: repositories: - repository: templates type: git name: CRHFCSIDevOps/mlife-common-templates Because a version is not specified in the example above, the mainline (develop in this case) will be used.

If you want to use a specific reference (e.g. a particular branch), you can do so by adding a ref as shown in this example:

resources:
 repositories:
 - repository: templates
 type: git
 name: CRHFCSIDevOps/mlife-common-templates
 ref: bugfix/39681-ignore-namespace-issues

It is also possible to create reusable steps within your own repository rather than using the common-templates maintained by the Energizers team. In this case you would create a sub-folder in your repo. In the example below the sub-folder is called parameters:

- stage: release_test
 dependsOn: build
 condition: and(succeeded(), ne(variables['Build.SourceBranch'], 'refs/heads/main'))
 jobs:
 - template: templates/aws-release.yaml
 parameters:
 awsCreds: mlife.dev
 awsBucket: mlife.dev.s3

References

  • https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/security/templates?view=azure-devops#set-required-templates
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/process/tasks?view=azure-devops&tabs=yaml
  • https://www.7pace.com/blog/azure-devops-best-practices
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/get-started/what-is-azure-pipelines?view=azure-devops
  • https://docs.microsoft.com/en-us/azure/devops/pipelines/security/templates?view=azure-devops
  • https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/add-template-to-azure-pipelines
  • https://git-scm.com/book/en/v2/Git-Basics-Tagging
  • https://docs.microsoft.com/en-us/azure/devops/repos/git/git-tags?view=azure-devops