Sprint 1.34.0 1/2 - Feb 21 to March 11

[marcushyett-ph]

Global Sprint Planning

Agenda:

• Team offsites - We're making this sprint 3 weeks long to account for some non-sprint priorities during the offsite.

Reminder: Launch week (21st March):

• Monday: Experimentation (100 completed experiments)
• Tuesday: "Project Diamond" (Improved quality, scalability ease of deployment and nailed trends)
• Wednesday: Taxonomy (% of active organizations with 5+ user taxonomized events or properties)
• Thursday: Collaboration (Increase discoveries of saved insights by 50%)
• Friday: Recordings - Primarily Marketing focussed: Launch new pricing for recordings and shout about the improvements we've already made

Weekly reiterating of company goals for 2022

• Quality: Our core features (insights, recordings, feature-flags / experimentation) work like a Swiss watch
• Extensibility: You can instantly connect PostHog to any other system
• Deployments: PostHog is the easiest self-hosted product to deploy and scale in the world

Retro: Status of Outcomes from Previous Sprint

1. Taxonomy. New data management page. Status: @alexkim205 Got started on routing but didn't get much done because of other taxonomic tasks (taggables and editable taxonomic definitions). Will spill into next sprint.
2. Audit log. Capturing the data, nothing social. Status: @pauldambra Mostly done for Feature Flags. I'm off for the rest of the week so would need handing over to someone or rolling on into next sprint
3. Collaboration. Sharing urls with project ID Status: @Twixes Short: Will likely go into the first half of the next sprint. Long: Fully shipping dashboard redesign and permissions crept into this sprint, so while the MVP enhancement to URLs is figured out, it's not built out yet.
4. Nail Trends: AND + OR filtering & Behavioural filtering (without optimizing performance) Status: @EDsCODE Progress on both backend handling and UI for filtering has been done. Most likely shippable for insights by the end of the week.
5. Increase ingestion robustness: followups for postgres outage & dumb events service & DLQ @yakkomajuri Status: great progress on plugin server reliability and observability. This is never-ending work but a lot was accomplished so this was a successful sprint. A bit of DLQ work but not much on the capture endpoint side.
6. Helm staging env - Ready to accept traffic Status: @guidoiaquinti (@hazzadous) Not achieved. Healthchecks added to ensure helm deployment 1. fails hard on incorrect config 2. services only go down on specific dependencies. Most time spend on postgresql monitoring and issue resolution. grafana + dashboards added to helm

Plan: Proposed Goals for Next Sprint

Each goal should have a single owner. Owner can only be an engineer.

1. Nailing Trends: Wrap up and apply OR filtering across entire App, Cohort behavioral filtering + addressing limitations with Feature Flags @EDsCODE
2. (nail) Project Homepage Summarize what's happening in your project to drive collaboration @rcmarron
3. Project Carnivore The name of the insight should be understandable @Twixes
4. Plugin Back-pressure (a.k.a Projech SHhhh) - Reduce the variability in back-pressure to eliminate pages @yakkomajuri
5. Postgres Cleanup - Close out P0s from Postgres issues @tiina303
6. ClickHouse - Upgrade and sharding plan (proof of concept) @guidoiaquinti & @hazzadous @macobo

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

Team ___

## Retro

<!-- Talk about what went well, what didn't go well and any actions to improve next time -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## Planning

<!-- Each item should have a single owner. Owner can only be an engineer. -->

### High priority

-

### Low priority / side quests

-

[mariusandra]

Team Core Experience

Retro

• Alex: tried to bite off a bit more than can chew. Many parallel PRs (8 at the peak). Takeaway: max at 3-4, work incrementally.
• Michael: pretty well, a lot of stuff done. Trouble with start and end of projects if they span sprints. The last 20% if sneaky. Takeaway: spec new work and leftovers out one level deeper (and at the right time).
• Paul: a lot of WIP, ended up doing 4 versions of the audit log and didn't finish. In a good place now. Takeaway: don't increase WIP too much when waiting for reviews
• Rick: good sprint, recording in funnels in. Cross team work. A lot of writing (Recordings infra, project home, ...). Negative: my internet is killing me.
• Chris: thinking of collaboration, not sure if it's going well. We could have gotten to decision points sooner instead of rehashing old ideas.
• Marius: reviewed a lot of WIP, offsite planning, etc. Didn't have time to water a few PRs of my own.

Hang over items from previous sprint

Still work left on:

• Taxonomy & data management (@alexkim205)
• Shareable URLs (@Twixes)
• Audit log (@pauldambra)

Planning

See parent issue.

[fuziontech]

Team Platform

Retro

• 🔥 s but we have a pretty decent grasp on next steps. Quite a bit of paging
• Tiina struggled with which of the two P0s are more important - async migrations and postgres follow up work. We'll document the concept of an unbreak now tag - @hazzadous wants spicy fly tag
• @yakkomajuri felt like the sprint went pretty well. A lot of progress made on async migrations. Tiina helped a ton there. Pulled the slack caused by the deep dives into the plugin server.
• @guidoiaquinti - Most of the sprint was spent working on stuff that was unrelated to what was planned. Brought up quite a bit of things that should be fixed. Not sure if we will actually fix all of them. Number 2 sprint where @guidoiaquinti really enjoyed working with @hazzadous
• @hazzadous - A bit hectic. Doesn't understand some parts of the system. Cohort / feature flagging - didn't see bigger picture. Needs more context on the product features. Spoke to Marcus and Paul about audit logs - enjoyed that discussion. It would be nice to have more cross team pollination (without force meetings) somehow make this organic.
• @marcushyett-ph - Very surprised at how we have gotten some real traction on the plugin bounty. Some follow ups on code quality and tests. Feels like he could have been more helpful with the cohort fire - maybe not offer behavioral cohorting.
• @timgl - remove papercups - customer support will own channels - pull in support hero when needed.

Last sprint's goals

• Increase ingestion robustness: followups for postgres outage & dumb events service & DLQ @yakkomajuri
• Helm staging env - Ready to accept traffic @guidoiaquinti (@hazzadous)

Hang over items from previous sprint

• Some postgres mitigation work is being carried over
• Some Plugin server work is also getting carried over
• Both because of 🔥

Planning

• Get plugin server back pressure metric so that it is not paging with current levels set @yakkomajuri
• Continuing Postgres mitigation steps P0s complete @tiina303
• Helm staging env - ClickHouse upgrade and sharding plan (proof of concept) @guidoiaquinti @hazzadous @macobo

High priority

P0s on Sprint Board

[EDsCODE]

Team Core Analytics

Retro

Eric:

• 7/10
• Fixed cohort recalculations
• OR filtering API nearing completion. Cleaning up related cases right now that need to be handled by filter properties

Neil:

• 6/10
• Why 6/10 ^
• OR filtering backend
• Event querying slowness fix

Li:

• 8/10 sprint ? overall pretty good, mostly achieved what the sprint goal was
• Or filtering ui
• Experiments secondary results - i haven’t forgot!!!

Harry:

• 4/10
• Added docs for removing indices
• Added pganalyze to heroku pg, not long term solution though but gives some insights (although not much above running a couple of queries on pg_* tables)
• Removed team_id index from persondistinctid to free up some space
• Not much aside from firefighting, but seems better now :fingercrossed:

Hang over items

Or property filtering

• Finish this on insights. Make sure it’s working for all insights
• Add ability to or filter on events page, person page, test accounts, cohorts etc

Planning

Cohort Behavioral Filtering

• Enumerate the possibilities we want for behavioral filtering
• Implement new UI to support these filters
• Backend logic for building these cohorts
• Address this limit we want to put on behavioral filters: Disable behavioral cohorts for new feature flags #8628

Misc:

• Close this: feat(trends): add moving average smoothing_intervals param #6435