3.10 release tracking issue

[beyang]

3.10 Release (2019-10-20)

At the start of the iteration

• Choose dates/times for the steps in this release process and update this issue template accordingly. Note that this template references working days, which do not include weekends or holidays observed by Sourcegraph.
• Add events to the shared Release Schedule calendar in Google and invite team@sourcegraph.com.
  • Creating the release branch.
  • Tagging the final release.
  • Publishing the blog post.
• Send message to #dev-announce with a link to this tracking issue to notify the team of the release schedule.
• Create the retrospective document and schedule the retrospective meeting within a few days after the release (send calendar invites to team@sourcegraph.com).
• Create a new test grid for MAJOR.MINOR by cloning the previous release testing grid on Monday.com and renaming it to "MAJOR.MINOR Release test grid".
  • Reset all tested cells to "To test", unless the "Automated" column is marked as "Done". See this article for how to update multiple values in Monday.com.
  • Assign rows in the release testing grid to engineers from the team that owns the row.
• Create reminders for yourself to preform the remaining sections in this checklist at appropriate times.

5 working days before release (2019-10-13)

• Use ./dev/release-ping.sh to message teammates who have open issues / PRs in the milestone and ask them to remove those that won't be done by the time that the release branch is scheduled to be created.
• Verify that there is a draft of the blog post and that it will be ready to be merged on time.
• Ping each team, and ask them to identify which of the optional rows that they own on the release testing grid should be tested this iteration.
• Ping the @distribution team to determine which environments each row on the release testing grid should be tested in.

4 working days before release (2019-10-14)

• HH:MM AM/PM PT Add a new section header for this version to the CHANGELOG immediately under the ## Unreleased changes heading and add new empty sections under ## Unreleased changes (example).
• Create the MAJOR.MINOR branch for this release off of the changelog commit that you created in the previous step.
• Tag the first release candidate vMAJOR.MINOR.0-rc.1:

  VERSION='vMAJOR.MINOR.0-rc.1' bash -c 'git tag -a "3.10" -m "3.10" && git push origin "3.10"'
  

• Send a message to #dev-announce to announce the release candidate.
• Run Sourcegraph Docker image with no previous data.
  • Run the new version of Sourcegraph.

    CLEAN=true IMAGE=sourcegraph/server:MAJOR.MINOR.0-rc.1 ./dev/run-server-image.sh
    

  • Initialize the site by creating an admin account.
  • Add a public repository (i.e. https://github.com/sourcegraph/sourcegraph).
  • Add a private repository (i.e. https://github.com/sourcegraph/infrastructure).
  • Verify that code search returns results as you expect (depending on the repositories that you added).
  • Verify that basic code intelligence works on Go or TypeScript.
• Upgrade Sourcegraph Docker image from previous released version.
  • Run the previous version of Sourcegraph.

    CLEAN=true IMAGE=sourcegraph/server:$OLDVERSION ./dev/run-server-image.sh
    

  • Initialize the site by creating an admin account.
  • Add a public repository (i.e. https://github.com/sourcegraph/sourcegraph).
  • Add a private repository (i.e. https://github.com/sourcegraph/infrastructure).
  • Stop the previous version of Sourcegraph and run the new version of Sourcegraph with the same data.

    CLEAN=false IMAGE=sourcegraph/server:MAJOR.MINOR.0-rc.1 ./dev/run-server-image.sh
    

  • Verify that code search returns results as you expect (depending on the repositories that you added).
  • Verify that basic code intelligence works on Go or TypeScript.
• Run the new version of Sourcegraph on a clean Kubernetes cluster with no previous data.
  • Create a new Kubernetes cluster using https://github.com/sourcegraph/deploy-k8s-helper.
  • Add a public repository (i.e. https://github.com/sourcegraph/sourcegraph).
  • Add a private repository (i.e. https://github.com/sourcegraph/infrastructure).
  • Verify that code search returns results as you expect (depending on the repositories that you added).
  • Verify that basic code intelligence works on Go or TypeScript.
  • Tear down this Kubernetes cluster.
• Delete entries from section 15 (CHANGELOG) of the testing grid, or move them into permanent sections above. Add new CHANGELOG items for this release into section 15. Assign the feature owner as the tester for each row.
• Send a message to #dev-announce to kick off testing day.
  • Include a link to the testing grid.
  • Include the command to run the latest release candidate:

    IMAGE=sourcegraph/server:MAJOR.MINOR.0-rc.1 ./dev/run-server-image.sh
    

  • Mention that testing is the top priority, it is expected to take the whole day, and that known or suspected regressions should be tagged as release blockers. Mention that, for other issues found, high-level details like customer impact should be added to help Product determine whether it's a release blocker.

3 working days before release (2019-10-15)

• Send a message to #dev-announce to report whether any release blocking issues were found.
• Add any release blocking issues as checklist items here and start working to resolve them.
• Review all open issues in the release milestone that aren't blocking and ask assignees to triage them to a different milestone (backlog preferred).

As necessary

• git cherry-pick bugfix (not feature!) commits from master into the release branch.
• Aggressively revert features that may cause delays.
• Re-test any flows that might have been impacted by commits that have been cherry picked into the release branch.
• Tag additional release candidates.

1 working day before release (2019-10-19)

• HH:MM AM/PM PT Tag the final release.

  VERSION='vMAJOR.MINOR.0' bash -c 'git tag -a "3.10" -m "3.10" && git push origin "3.10"'
  

• Send a message to #dev-announce to announce the final release.
• Verify that all changes that have been cherry picked onto the release branch have been moved to the appropriate section of the CHANGELOG on master.
• Wait for the final Docker images to be available at https://hub.docker.com/r/sourcegraph/server/tags.
• In deploy-sourcegraph:
  • Wait for Renovate to open a PR to update the image tags and merge that PR (example).
  • Create the MAJOR.MINOR release branch from this commit.
  • Tag the vMAJOR.MINOR.0 release at this commit.

    VERSION='vMAJOR.MINOR.0' bash -c 'git tag -a "3.10" -m "3.10" && git push origin "3.10"'
    

• Open (but do not merge) PRs that do the following:
  • Update the documented version of Sourcegraph (example).

  find . -type f -name '*.md' -exec sed -i '' -E 's/sourcegraph\/server:[0-9\.]+/sourcegraph\/server:MAJOR.MINOR.0/g' {} +
  # Or use ruplacer
  ruplacer --go -t md 'sourcegraph/server:[0-9\.]+' 'sourcegraph/server:MAJOR.MINOR.0'
  

  • Update latestReleaseKubernetesBuild and latestReleaseDockerServerImageBuild (example).
  • Update deploy-aws version
  • Update deploy-digitalocean version
  • Message @slimsag on Slack: MAJOR.MINOR.PATCH has been released, update deploy-sourcegraph-docker as needed
  • Update versions in docs.sourcegraph.com header (example)
• Review all issues in the release milestone. Backlog things that didn't make it into the release and ping issues that still need to be done for the release (e.g. Tweets, marketing).
• Verify that the blog post is ready to be merged.

By 10am PT on the 20th

• Merge the docs PRs created in the previous step.
• Merge the blog post (example).
• Close this issue.
• Close the milestone.
• Notify the next release captain that they are on duty for the next release. Include a link to this release issue template.
• Remind the team that they should submit retrospective feedback 24 hours before the scheduled retrospective meeting.

After the Retrospective

• Scrub the retrospective Google doc for any priviledged customer data.
• Convert to Markdown.
• Add a new retrospective page in sourcegraph/doc/dev/retrospectives.
• Add a link to it on retrospectives/index.md and in the left nav (sourcegraph/doc/_resources/templates/doc.html)