Sprint 1.29.0 1/2 - Sep 6 to Sep 17

[marcushyett-ph]

Based on the new structure for sprint planning on weds https://github.com/PostHog/company-internal/issues/401

Retro:

• Finish Nailing Funnels
  • By Friday - funnels will be done.
• Get started with paths
  • Ton of querying work to unblock use cases
  • Design started
• Validate approaches to quant analysis
  • Validated general premise and approaches
  • Identified concrete solutions
  • Identified performance and scaling challenges
• Performance improvements
  • Tactical customer performance mitigation
  • Materialize cols automation

I wanted to propose a number of priorities we might consider (and specific outcome) - we'll discuss in the meeting today but feel free to add - any other priorities you propose:

• Nail Paths User Experience - Core Analytics
  • Outcome:
    • Get draft designs in-front of users and feedback?
    • Work out where they went after they dropped off (e.g. Rageclick, PageLeave another event or action, another URL)
    • Work out how they got to the step where they dropped off (e.g. another product, the previous funnel step, an external URL)
    • I need to see more than 3 steps to reach a conclusion (infinite ideally, perhaps 5 or 6 might be enough)
    • I'd like to understand conversion rates to outcomes as percentages rather than absolute numbers
    • I'd like to see the path of a cohort already implemented
    • I'd like to see the path between two events
    • I'd like to flexibly filter out noisy events
  • What happened:
    • Built API for all above Use Cases
    • Design out - iterating based on user feedback
    • Scope increased based on feedback to include wildcarding
• Enterprise Unblock Project Based Permissioning: Core Experience
  • Outcome: Project-based permissioning #4263 P0's shipped
  • What Happened:
    • Documentation done
    • Structured done
    • Won't get it shipped this sprint - code will be ready by end of sprint, getting the code reviewed will take a few days more - we need to make it REALLY secure - so we need to pay more attention
• Performance - Core Analytics + Platform
  • Outcome: Figure out improvements without changing the software (Plan to spin CH out of K8s, Scaled out CH on K8s as a test)customers?
    • Validate and stabilize impact of materialized cols
    • Significant improvement in funnels performance (may be resolved by materialize cols)
  • What Happened:
    • Materialized Properties used in trends and funnels
    • Tooling to see how queries are performing
    • More performance out of current stack - continue to work on splitting out CH from Helm chart
• Duplicate persons and distinct IDs & Cohorts (https://github.com/PostHog/product-internal/issues/126) Platform
  • Outcome: We have worked out why this issue is happening
  • What Happened:
    • Only 3 days of work (with vacations and support hero)
    • Narrowed down in some race conditions that were problematic
    • Ruled out issues with Kafka and other systems
    • Pushing large fixes this week - with monitoring
• Quantitative Analysis Design: Design + Core Analytics
  • Outcome: We have feedback from multiple users on design drafts
  • What Happened:
    • No progress pushed focus deeper into funnels

[macobo]

Performance and reliability wise I think the goal should be to speed up funnels to an acceptable level.

Funnel timeouts are an issue still.

8% of all funnel queries fail over the last 30 days. Breakdown of reasons:

80% of these failures are performance-related, 20% bugs.

For fixing these:

• Finish up work on materialized columns.
  • Some work for materialized person properties usage will spill over to next sprint.
  • Fine-tuning, measuring, etc to make sure
  • Expectation: this should reduce timeouts significantly
• Start introducing funnels-specific speedups - e.g. reducing number of subqueries, etc.
• Make sure Vendesta has a better experience performance-wise: both re hardware as well as schema-wise.

Personal note: I will likely be out one of the weeks after the offsite.

[samwinslow]

Growth

Retro:

• Continued case studies and partnership-building — to get closer toward understanding the commonalities between our existing customers & assist content creation
• Website updates
  • Shipped Blog layout
  • Shipped Font hosting & delivery
  • Designed new in-app Signup page
  • Docs improvements
• Discussing tactics to work with external marketing agency, including GTM alternatives
• API work for multivariate flags, and UI PR open here: Multivariate flags UI #5725

Forward-looking:

• Propagate design changes from new Handbook layout to rest of website
• Finish multivariate flag support as an internal tool (this week)
  • Then: test homepage changes, especially copy changes, for conversion optimization and also dogfooding the tool
• Solidify a partnership plan with n8n (and others?), launch some case studies
• Make necessary changes to interface with marketing agency

Notes:

• Sam OOO week after the offsite (until Thurs 9/9)

[fuziontech]

Platform

Retro:

• Great sprint! We got person_distinct_id to collapsing merge tree
• All P0s are mostly done
• Hired a great SRE candidate
• I was support hero last week so 33% of the team wasn't able to focus 100% on shipping things, but most of this sprint has been support anyways with charts for customers.
• @yakkomajuri on the team is huge win a lot of good discussion on export plugins and how to consume data into PostHog
• Got a lot of softer stuff done around discussions re: import and export plugins
• Tiina thought it was 👍 bunch of random stuff

For the next sprint:
P0

• General on prem support
• Gotta go fast. Measuring and delivering performance improvements. Now that our chart is 99% there we need to work on cloud specific improvements and docs for scaling the chart.
• Support for on-prem deploys and collecting important telemetry there.

P0.5

• Start automating ClickHouse not on K8s for Cloud and for large installs
• Test scaling out ClickHouse with ClickHouse operator
• Replicating Postgres into ClickHouse

Plugins:
posthog/plugin-server#547

Notes:

• Jams OOO on and off for a few days 2 weeks after offsite

[marcushyett-ph]

@EDsCODE @clarkus plan out specific outcomes for paths and update the comment at the top with the specific outcomes for the next sprint

[marcushyett-ph]

Please add your team plans here!

[mariusandra]

From the priorities above, only #4263 falls directly on Team Core Experience.

Thus we have a bit of capacity to clean house this sprint, which includes (bugs and) getting through the door a few small features that are designed, but not yet implemented. List below.

• Plans split per person:

  • Michael: Project-based permissioning #4263 and related API changes (e.g. Allow for multiple projects to be viewed at the same time #5363)
  • Chris: create product designs that spark joy (project based permissioning, for other teams: changes to paths, quantitative analysis etc) + adopting storybook to track implemented designs
  • Paolo: usability tests for paths and/or quantitative analysis, supporting everyone in product work, PR reviews, SAML if something still to add Full SAML login support #5647
  • Alex: one week off, then continue on saved insights from Li, and/or work on features already designed by Chris (list below)
  • Marius: 33% code reviews + 33% pairing + 33% miscellaneous bugs and work to unblocking others, if time implement any of the designed features from below

• Already designed, but not implemented changes:

  • Improve UX for formulas in trends #3375
  • Standardized error formatting for form elements #5380
  • Make compare previous great! #4705
  • Legend key in chart #4594
  • Better tooltips for filtered trends #5229 - would be a really nice change to make
  • Nicer errors when user entered an invalid regex #4680

That is, unless the offsite reveals deeper work we need to be doing instead.

[EDsCODE]

UI Implementation

New paths UI and all the features/toggles below

• Work out where they went after they dropped off (e.g. Rageclick, PageLeave another event or action, another URL)
• Work out how they got to the step where they dropped off (e.g. another product, the previous funnel step, an external URL)
• Path depth toggle
• Conversion rates as percentages and absolutes
• Time between steps
• Filter out events as needed
• Filter Start point/End point
• Funnel search capability -> Expand the paths between funnel steps to see how users got between steps

Performance improvement

• Finish up work on materialized columns.
• Some work for materialized person properties usage will spill over to next sprint.
• Fine-tuning, measuring, etc to make sure
• Expectation: this should reduce timeouts significantly
• Start introducing funnels-specific speedups - e.g. reducing number of subqueries, etc.
• Make sure Vendasta has a better experience performance-wise: both re hardware as well as schema-wise.

[paolodamico]

This sprint is over