Release Manager Runbook

Release process overview

Here’s a quick review of what the release process looks like from the community perspective:

• This is the expected release cadence .
• The release calendar is awesome. It gives you an agenda with the expected duties.
• Here’s what the project expects all contributors to do for backports/patches .

If the builds break, you can take a look at some common issues to see if we’ve encountered them before.

Verify you have access

If you don’t have access to any of the following, contact a member of the TOC or SC.

• You’re a member of the release-managers@spinnaker.io group and a manager of the spinnaker-announce@googlegroups.com group. (You’ll get a permissions error on those pages if you don’t have access.
• You’re a member of the release-managers GitHub team .
• You’re able to view our GCP spinnaker-community cloudbuilds . You should see a lot of builds.

Build Artifact Table

See the GitHub Action files in each service’s git repository.

One week before the branches are cut

Ping #dev reminding everyone to merge outstanding changes by <DAY>:

The release manager will be cutting the $VERSION release branches next , so if there are any outstanding PRs that you’d like to get into $VERSION, please make sure they are merged by EOD next <DAY - 1>. Once the branch is cut, only fixes will be accepted into the release branches.

The day the branches are cut

1. If there are any outstanding autobump PRs , make the required fixes to allow them to merge. (You can ignore keel and swabbie; those repositories aren’t part of a Spinnaker release.)

2. Tag repositories with their respective next semVer minor version:

   1. Tag HEAD of master in kork if required, merge autobump PRs.
   2. Tag HEAD of master in fiat with v{major}.{minor}.0, merge autobump PRs.
   3. Tag HEAD of master in orca with v{major}.{minor}.0, merge autobump PR in kayenta.
   4. Tag HEAD of master in remaining repos with {major}.{minor}.0.
   5. Don’t tag spin repository. We’ll tag after branch cut.

   The BOM will become these tags.

3. Create the release branches by running buildtool new_release_branch

   1. Create the spin release branch at HEAD of master. Later we will tag our new branch with the Spinnaker Release version which creates our artifacts.

4. Ping #dev with some version of this message.

   The release branches for Spinnaker $VERSION have been cut from master! Those branches are only accepting fixes for existing features. Please post in this channel and tag me @$YOUR_NAME if you would like a fix cherry-picked into the release. If you would like to highlight a specific fix or feature in the release’s changelog, please make a pull request against the curated changelog by Friday.

One week after branches are cut

1. Audit backport candidates .

2. FIXME: Run buildtool integration tests in Google Cloud Build to validate artifacts

   TODO: We may need a BOM or at least a list of artifacts to run integration tests with.

3. Iterate until integration tests pass; fixing issues, tagging repositories and running the integration tests.

4. Ensure bumpdeps have completed and gradle versions have propagated.

5. Run buildtool Release GitHub Action , with dry run true.

6. Inspect job, check Cat output files for review step, validate BOM, changelog, and versions.yml look correct.

7. Run buildtool Release GitHub Action , with dry run false.

8. Use regctl to add tag’s to both slim and ubuntu containers - see buildtool release_helper.sh

   1. at {tag} WITHOUT unvalidated, For example: tag orca:1.2.3-unvalidated with orca:1.2.3. Halyard expects container tags and debian packages to match the BOM.

   2. at release-{major}-{minor}-{patch} - friendly tag for use by Spinnaker users Kubernetes and other manifests. For example: spinnaker-1.27.0

9. Update Next Release Preview.

10. Seek review and merge of the spinnaker.io docs PR created by buildtool.

11. Ping the #spinnaker-releases channel to let them know that a new release is available.

    > Hot Tip! You can use giphy to tell everyone it's released!
    >
    > `/giphy #caption "Spinnaker {VERSION} has been released!" gif search query`
    

12. Publish a Spin CLI minor version.

    1. Each Spin CLI release is tied to a version of Gate. To ensure compatibility, regenerate the Gate Client API.

    2. Follow the instructions to update the gate client.

    3. Ensure that the GitHub Action’s for the above PR merge were successful and then push a git tag for the release version 1.{minor}.{patch} to the spin repository. This will kick off binary and container build & push to GCS and GAR.

Patch a previous Spinnaker version

Repeat as needed for each supported version.

1. Audit backport candidates .

2. Ensure all branches have tags at HEAD or a suitable point.

3. The rest of the process is almost the same as above steps for releasing a new minor version.

   Start with ensuring CI is passing on the branches, bumpdeps completed, do a GHA dry run, check output, etc.

4. Ping the #spinnaker-releases channel to let them know that the new patch is available.

   > Hot Tip! You can use giphy to tell everyone it's released!
   >
   > `/giphy #caption "Spinnaker {VERSION} has been released!" gif search query`
   

5. Approve the spinnaker-announce email (link will come in email). You can approve the message in the spinnaker-announce group .

Release minor-version Halyard

Repeat as needed.

1. Check for outstanding PRs.

2. Tag master with next {minor} tag. For Example v1.45.0 next minor is v1.46.0. This will result in a new debian package at 1.46.0 and a new containers tagged 1.46.0-unvalidated(-ubuntu).

3. Create release-{major}-{minor}-x branch at the new tag. This enables backports and {patch} releases to be made.

4. TODO: Any validation?

5. Re-tag Halyard container without -unvalidated. halyard repository

6. Re-tag latest stable Halyard containers with :stable and :stable-ubuntu. docs usage

7. Post in #halyard that a new version of Halyard has been released.

   Hot Tip! You can use giphy to tell everyone it’s released!

   /giphy #caption "Halyard {VERSION} has been released!" gif search query

Release patch-version Halyard

Repeat as needed.

1. Ensure you have audited all Halyard backport candidates .

2. Tag release-{major}-{minor}-x branch with next {patch} tag. For example v1.46.0 next tag is v1.46.1

3. TODO: Any validation?

4. Re-tag Halyard container without -unvalidated. halyard repository

5. Re-tag latest stable Halyard containers with :stable and :stable-ubuntu. docs usage

6. Post in #halyard that a new version of Halyard has been released.

   Hot Tip! You can use giphy to tell everyone it’s released!

   /giphy #caption "Halyard {VERSION} has been released!" gif search query

Publish a new version of deck-kayenta

Repeat as needed.

Follow the instructions in deck-kayenta’s README .

Audit backport candidates

Repeat weekly.

1. Audit each PR that has been labelled a backport candidate .

2. If a candidate meets the [release branch patch criteria]({{> ref “releasing#release-branch-patch-criteria” >}}):

   1. Remove the `backport-candidate` label from the PR.
   
   1. Determine which versions the PR needs to be backported to. If it gets backported to an older version, all new versions should get the backport as well. Go only as far back as the supported [stable versions](/docs/releases/versions/#latest-stable).
   
   1. Add a comment instructing
      [Mergify](https://doc.mergify.io/commands.html#backport) to create
      backport PRs against one or more release branches. For example, to
      create backport PRs against the 1.19, 1.20 and 1.21 release branches, comment:
   
      > @Mergifyio backport release-1.19.x release-1.20.x release-1.21.x
   
   1. Approve and merge the backport PRs.
   
   1. If Mergify cannot create a backport because there are merge conflicts,
      ask the contributor to open a PR against the target release branches with
      their commits manually
      [cherry-picked](https://git-scm.com/docs/git-cherry-pick).
   

3. If a candidate does not meet the release branch patch criteria , add an explanation to the contributor as a comment.

   1. If it's impossible for the candidate to meet the criteria (for example, it doesn't
      fix a regression), remove the `backport-candidate` label.
   
   1. If the contributor can amend the candidate to meet the criteria (for example,
      add test coverage), don't remove the `backport-candidate` label.