Sprint Sep 19 - Sep 30

[timgl]

Global Sprint Planning

3 things that might take us down

Last week

1. Rate limiting API requests - @rcmarron to figure this out, just target clickhouse

Retro: Status of Outcomes from Previous Sprint

can only be an engineer.

1. Add session recording billing @kappa90 -> working on it, scope has increased a bit, but we'll end up with a future proof system.
2. Continue bugbashing. @mariusandra to provide a list on Friday. Timebox to this sprint https://github.com/orgs/PostHog/projects/58/ -> 8 more bugs done, 5 in progress, 2/3 finish this sprint. They're big bugs.
   why: Make sure we have a great experience + increase retention
3. $create_alias deprecation @yakkomajuri -> instrumented metrics, talked to users. Lots of users using this.
   why: to roll out persons on events
4. PostHog EU (ClickHouse / Zookeeper production ready, runbooks, + other services if possible) @ellie -> up and working! Small things to iron out, load testing, then get beta testers on next week!
   why? to increase usage from customers that need EU cloud
5. Harry: investigate latency issues and random error rate increase @hazzadous -> Resolved.
6. Session recording event log filtering in playlist @alexkim205 -> No progress. Will get this out next sprint.
   why? Make our session recording experience comprehensive/competitive
7. Complete person on events rollout @EDsCODE. -> backfilling for person/group properties didn't work. Digging into this.
   why? To unlock scale & quicker queries
8. Feature Flags: posthog-js and react-native bootstrapping, server side override, GeoIP overrides for everyone, and UX issues: audit history, feature flags on person @neilkakkar -> the whole list is sorted.
   Why? To make our feature flag competitive

Retro: What can we do better next sprint?

Support hero this sprint

Week 1: Karl
Week 2: Li

Plan: Proposed Goals for Next Sprint

Each goal should have a single owner. Owner can only be an engineer.

1. Text tiles on Dashboards with our team and beta users @pauldambra
   Why? People asked for this
2. re-re-re-naming calculated events @mariusandra
   why? Ongoing confusion, Annika has done good work so we can solve this
3. Debug and re-run persons on events backfill @yakkomajuri (for now)
   why? To speed up queries
4. Deprecate $create_alias @yakkomajuri
   why? So we can release persons on events
5. 🇪🇺 EU plan and build out testing framework for load for new infrastructure (both ingestion and query side) @ellie
   why? New infra so we want to test this
6. Continue improving visibility and making loki logging more scalable @hazzadous
   why? Because Loki isn't reliable, because we have problems that we can't debug.
7. Continue supporting person on events rollout where possible @EDsCODE
   why? To speed up queries
8. List of clean up tasks for team west @liyiy @neilkakkar @EDsCODE
   why: very little capacity based on schedules
9. Getting new PlaylistView out for session recordings @alexkim205
   why? so people can binge more recordings
10. PoC of Billing service @kappa90
    why - Getting Billing right ASAP will start unlocking revenue for SeshRec™️

Team sprint planning

For your team sprint planning copy this template into a comment below for each team.

Team ___

## Retro

<!-- Talk about what went well, what didn't go well and any actions to improve next time -->

- 

## Hang over items from previous sprint

<!-- For each item, decide to re-prioritise (and add below) or deprioritise -->

- Item 1. prioritised/deprioritise

## Planning

<!-- Each item should have a single owner. Owner can only be an engineer. -->

### High priority

-

### Bugs

-

### Low priority / side quests

-

[pauldambra]

Team East

Retro

Hang over items from previous sprint

Continue bugbashing. @mariusandra to provide a list on Friday. Timebox to this sprint https://github.com/orgs/PostHog/projects/58/
why: Make sure we have a great experience + increase retention

• Lemon Tables (possibly) @mariusandra
• Annotations @Twixes

Planning

We have one week of the sprint and then offsite week in Rome

High priority

• Text tiles on Dashboards with our team and beta users @pauldambra
  • why: users asked for it, we believe it makes dashboards more useful, and so we'll see increase in stickiness
• re-re-re-naming calculated events @mariusandra
  • why: we've only get a week until the offsite, Alex should focus on session recording off-boarding, we believe we can resolve this issue in a week
  • see https://github.com/PostHog/product-internal/issues/344

Bugs

• calculate event property usage @pauldambra
  • should be fixed (enough) in the next couple of days and then only need periodic monitoring

[yakkomajuri]

Team Ingestion

Retro

Goal: Stay afloat + create_alias deprecation

• Stayed afloat
• Set up metric for create_alias, spoke to some teams, thought about how we might inform users, but didn't go further
• Was informed that the backfill for persons on events didn't work correctly. How could we have prevented finding this out so late?
  • Lessons:
    • Have a post-migration check
    • Double check everything, even things that "cannot possibly go wrong"
    • How much context should be shared with non-owners

Goals

1. Kafka queue for jobs @yakkomajuri
2. Debug and re-run persons on events backfill @yakkomajuri (for now)

[fuziontech]

Team Infrastructure

Retro

Harry - Events in EKS! 🥳 It's been a struggle trying to keep the infra from collapsing (logging). Firefighting bits and bobs. Everything is a blur. What did we even do?
Yakko - Goal: stay afloat. Outcome: Still dry. Didn't have anyone to work with for spec-ing which was not idea. Person on events has a few bugs that need to be investigated. How deep into the weeds should he go into to make sure everything goes perfectly
Ellie - PostHog EU IS SHIPPED!! 🚀 Logging is working as well which is amazing. Sounds like a great sprint
Guido - Mainly worked on PostHog EU. Very happy with working with Ellie and Harry. Generally happy. So far support hero has been ok, but it's getting a bit busy.

Important bits

• Guido out for this sprint
• Ellie out a few days

Hang over items from previous sprint

• There are still some PG and CH things going on. Not gotten to the bottom of issues, but at least have more visibility into it.

Planning

• 🇪🇺 EU plan and build out testing framework for load for new infrastructure (both ingestion and query side)
• 🇪🇺 Finalize list of Beta users for EU infrastructure
• Continue improving visibility and making loki logging more scalable.
• Set Q4 goals
• Ongoing: Customer support

[EDsCODE]

Retro:

• Person on events
  • Attempted rollout
  • Lifecycle wasn’t working
  • Data quality issue with person/group properties + created_at on events
    Feature Flags
• Backend/Client library updates—everything in review
• UX improvements—QA

Planning:
Li on support the second week
Eric doing release on the second week
Neil off first week + half of second week

• Continue supporting person on events rollout where possible
• Join team east bug squashing adventure
  why: very little capacity based on schedules

Related work:
- Currently in progress person on events PRs
- Project settings revamp
- Several cohort bugs that have been lingering
- Fixed date range filtering for cohorts
- Recordings + toolbar domains are combined but this could be split in settings
- Date property handling in all libraries for local evaluation
- User interviews on new feature flag improvements

[benjackwhite]

Team SeshRec Recordings

Retro

Hang over items from previous sprint

• Getting new PlaylistView out - prioritise
  • filtering UI
  • console logs

Planning

1. Off-board Ric successfully! @benjackwhite
   why - This is our last opportunity to learn from the master 😍
2. PoC of Billing service @kappa90
   why - Getting Billing right ASAP will start unlocking revenue for SeshRec™️

High priority

• Getting new PlaylistView out - prioritise
  • filtering UI
  • console logs

Bugs

• Yes there are lots

Low priority / side quests