Skip to content
This repository has been archived by the owner on Sep 30, 2024. It is now read-only.

Distribution: 2020.10.20 - Vibrant Beetle #14408

Closed
26 of 36 tasks
pecigonzalo opened this issue Oct 5, 2020 · 10 comments
Closed
26 of 36 tasks

Distribution: 2020.10.20 - Vibrant Beetle #14408

pecigonzalo opened this issue Oct 5, 2020 · 10 comments

Comments

@pecigonzalo
Copy link
Contributor

pecigonzalo commented Oct 5, 2020

Plan

Support new and existing deployments

This is an ongoing expense, we anticipate this taking no more than 5d of work spread across the entire team.

Create a Dhall demo to gather customer feedback

We would like to create a demo that we can share with customers and gather feedback about our proposed solution.

Releases are created in a single day

We have a goal of reducing the time it takes to create releases, and this current several-day system has encouraged us to view releases as “baked” rather than “snapshots of the main branch”, leading to situations where main is broken and we have to retrospectively fix it or adding last minute features.

Availability

Period is from October 20th to November 2nd (10 working days). Please write the days you won't be working and the number of working days for the period.

Tracked issues

@bobheadxi: 7.00d

  • deploy-sourcegraph: mark automated PR failures as release-blockers (#14622) 0.50d
  • qa: automated sourcegraph/server upgrade test (#13877) 3.00d
  • set up automerge for third-party license updates (#14766) 0.50d 🎩

Completed: 3.00d

  • (🏁 12 days ago) yarn release publish does not behave as expected (#14925) 🐛
  • (🏁 9 days ago) release: releases steps should happen as fast as possible (#14911) 0.50d
  • (🏁 8 days ago) release: generated tracking issue has incorrect dates when run from non-PST time zone (#14626) 0.50d 🐛
  • (🏁 5 days ago) release: explore using campaigns to manage release PRs (#14971) 1.00d 🎩
  • (🏁 4 days ago) release: announce that a release is out (#14912) 0.50d
  • (🏁 4 days ago) Walk Robert and Dax through managed instance upgrade process (#14998)
  • (🏁 3 days ago) sourcegraph/customer (#97) 0.50d 👩

@davejrt

  • sourcegraph/customer (#111) 👩

Completed

  • (🏁 130 days ago) Bigdata customer Tracking issue (#11717)
  • (🏁 5 days ago) sourcegraph/customer (#122) 👩
  • (🏁 5 days ago) sourcegraph/customer (#115) 👩
  • (🏁 4 days ago) blackbox exporter & site 24/7 next steps (#13627) 🧶
  • (🏁 3 days ago) Run QA tests on bare-metal Buildkite agents on every commit to master (non-blocking) (#12340)

@daxmc99: 4.00d

  • Automate Kubernetes cluster QA tests (#13878) 4.00d

Completed

  • (🏁 13 days ago) Add PG_HERO for Cloud SQL (#14865)
  • (🏁 7 days ago) RFC 174: HA Postgres for Sourcegraph Cloud (#11496)
  • (🏁 4 days ago) Walk Robert and Dax through managed instance upgrade process (#14998)

@efritz

  • docs: Update pure-docker upgrading docs for 3.21 (#14671) :shipit:

@ggilmore: 5.00d

  • Create customer demo video for dhall direction (#14646) 5.00d
  • ci: build and pin tool apks in CI for release (#13297) 🧶

Completed

  • (🏁 4 days ago) sourcegraph/customer (#74) 👩

@pecigonzalo: 1.00d

Completed: 1.00d

  • (🏁 5 days ago) Cancelled jobs in buildkite leave vagrant boxes running (#14582) 1.00d
  • (🏁 4 days ago) blackbox exporter & site 24/7 next steps (#13627) 🧶

@slimsag: 0.50d

  • sourcegraph/customer (#119) 👩

Completed: 0.50d

  • (🏁 7 days ago) Solidify plan for Stephen ownership handoff (#14864)
  • (🏁 6 days ago) sourcegraph/customer (#84) 👩
  • (🏁 5 days ago) sourcegraph/customer (#122) 👩
  • (🏁 5 days ago) sourcegraph/customer (#120) 👩
  • (🏁 4 days ago) sourcegraph/customer (#103) 👩
  • (🏁 4 days ago) Walk Robert and Dax through managed instance upgrade process (#14998)
  • (🏁 4 days ago) sourcegraph/customer (#74) 👩
  • (🏁 today) update 1password QA test env vars (#14875) 0.50d

@uwedeportivo: 5.00d

  • Create customer demo video for dhall direction (#14646) 5.00d

Legend

  • 👩 Customer issue
  • 🐛 Bug
  • 🧶 Technical debt
  • 🎩 Quality of life
  • 🛠️ Roadmap
  • 🕵️ Spike
  • 🔒 Security issue
  • :shipit: Pull Request
@pecigonzalo pecigonzalo added this to the 3.22 milestone Oct 5, 2020
@pecigonzalo pecigonzalo changed the title WIP: Distribution 3.22 Tracking issue WIP: Distribution 3.22/1 - Vibrant Beetle Oct 12, 2020
@pecigonzalo pecigonzalo changed the title WIP: Distribution 3.22/1 - Vibrant Beetle WIP: Distribution 3.22 - 2020.10.20 - Vibrant Beetle Oct 12, 2020
@pecigonzalo pecigonzalo changed the title WIP: Distribution 3.22 - 2020.10.20 - Vibrant Beetle WIP: Distribution 3.22 - 2020.09.20 - Vibrant Beetle Oct 12, 2020
@pecigonzalo pecigonzalo changed the title WIP: Distribution 3.22 - 2020.09.20 - Vibrant Beetle WIP: Distribution 3.22 - 2020.10.20 - Vibrant Beetle Oct 12, 2020
@pecigonzalo pecigonzalo changed the title WIP: Distribution 3.22 - 2020.10.20 - Vibrant Beetle WIP: Distribution - 2020.10.20 - Vibrant Beetle Oct 15, 2020
@pecigonzalo pecigonzalo modified the milestones: 3.22, Distribution: 2020.10.20 - Vibrant Beetle Oct 15, 2020
@pecigonzalo pecigonzalo added this to the Dist: 2020.10.20 milestone Oct 15, 2020
@pecigonzalo pecigonzalo changed the title WIP: Distribution - 2020.10.20 - Vibrant Beetle Distribution - 2020.10.20 - Vibrant Beetle Oct 19, 2020
@daxmc99
Copy link
Contributor

daxmc99 commented Oct 24, 2020

This week:

  • Finish up remaining Cloud SQL issues and sunset Postgres deployment.
  • Added Postgres users for every deployment
  • Discussed Deployment UX

Next week:

@ggilmore
Copy link
Contributor

ggilmore commented Oct 26, 2020

This week:

  • Retrospective
  • Deployment Discussion
  • Prepared all prerequisite materials for recording the customer demo.
    • Transformation of old dhall poc into something that's compatible with ds-to-dhall (to be able to show off incremental diffing process for migration) took the most time
    • Prepared "fake" dhall documentation for video
    • Prepared upgraded version of old dhall poc (with 3.21 images) to show off simple upgrade process (just change the URL)

Next week:

@bobheadxi
Copy link
Member

bobheadxi commented Oct 26, 2020

Last week

Some handbook updates (deployments, prometheus), improvements to alert solutions documentation rendering. Made more improvements to releases: reduced timeline, date formatting, etc. Took a stab at continuous sourcegraph upgrade tests (https://github.com/sourcegraph/sourcegraph/pull/14974) and using campaigns in releases (https://github.com/sourcegraph/sourcegraph/pull/15032)

This week

Continue working on https://github.com/sourcegraph/sourcegraph/pull/14974 and land sourcegraph/about#1838 . Turns out the automerge for license updates isn't working out - gonna circle back to that and see if that's easy to fix: https://github.com/sourcegraph/sourcegraph/issues/14766#issuecomment-716486279

@pecigonzalo
Copy link
Contributor Author

Last week

I have cleanup unused AWS resources and reviewed a couple of RFCs (RFC-249, RFC 245) and PRs. I have also been working with CE on the support pipeline.

This week

Im working on https://github.com/sourcegraph/sourcegraph/issues/13627 porting the checks to blackbox-exporter and merging the improved deployment pipeline goal and planning the next sprint.

@davejrt
Copy link
Contributor

davejrt commented Oct 30, 2020

Last Week

A lot of focus and firefighting with the release. Getting this into the hands of capital one and helping them with a small hiccup due to not following the upgrade path.

Closing out sourcegraph/sourcegraph#12340 and breaking that down into smaller issues for each failing component, however the infrastructure in place can now be used more generically for other tests

This week

Continue work on the failing regression/qa tests as well as help Robert and Dax with their efforts to test upgrades and k8s deployments.

@emidoots
Copy link
Member

This week

Pushed through as much hand-off as I can, mostly spent time documenting things and walking others through various processes. Up-to-date project board with where I am at here.

Next week

I expect to finish all hand-off tasks I have and close out that project before heading out to go on vacation for two weeks. Aiming to set myself up with as clean a slate as possible for when I return. I also hope to start work on my first high-impact, low-cost win by starting to investigate reducing the time it takes to loadd/display search results.

@ggilmore
Copy link
Contributor

ggilmore commented Oct 31, 2020

This week:

I task switched and spent most of my time on https://github.com/sourcegraph/customer/issues/74. After that, I then recorded and released released a rough draft of the Dhall demo video (https://github.com/sourcegraph/sourcegraph/issues/14646) for @sourcegraph/distribution to review in https://sourcegraph.slack.com/archives/CJX299FGE/p1604110703335500.

Next week:

I will respond to feedback on the rough draft Dhall video that I shared, and I aim to restart work on https://github.com/sourcegraph/sourcegraph/issues/13340

@bobheadxi
Copy link
Member

This week

Finalized and landed campaigns in releases and "closing" a release. Performed some instance upgrades for $CUSTOMER, made some progress on sourcegraph/server upgrade tests, made handbook updates for releases and managed instances. Opened a PR cleaning up how versions are configured in the release tooling (sourcegraph/about#1880).

Next week

Plan issues for 11.03 work (release tooling + k8s.sgdev marketing). Try to wrap up issues kicked from this sprint to 11.03 (sourcegraph/server upgrade tests, automerge).

@daxmc99
Copy link
Contributor

daxmc99 commented Nov 2, 2020

This week
Stephen guided Robert and me through a managed instance upgrade process.
Performed some debugging of our services in production to determine the root-cause of our repo-updater service failing to start (link). Investigated performing automated QA tests on our K8s cluster, similar to how we perform e2e & regression tests on our server and docker-compose deployments.

Next Week
Perform several managed instance upgrades. Begin running QA tests on sourcegraph/sourcegraph targetting K8s. Complete some small innovation tasks around using containerd images on our production clusters and evaluating ArgoCD & Flux as part of the Improve deployment pipeline UX distribution team goal.

@pecigonzalo
Copy link
Contributor Author

Last week
I merged the port of the site24x7 metrics to blackbox-exporter and was able to merge the Pipeline UX goal, but I forgot to add a problem statement. I have sent a new PR to add one. I have also had a meeting with CE to sync on how to handle CE <-> Eng workflow going forward and as we scale.
I also did a bit cleanup, removing the old ops_team OpsGenie group.

This week
The focus this week is to update our team goals and roadmap, referencing the items that are still relevant from sourcegraph/about#1104. Similarly I will get started on a draft around team growth ideas, where we want to be in the future, what is working, what is not, and the steps we need to take to get there.

@pecigonzalo pecigonzalo changed the title Distribution - 2020.10.20 - Vibrant Beetle Distribution: 2020.10.20 - Vibrant Beetle Nov 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

7 participants