Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outline new structure for experiments #9938

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
+68 −0
Draft

Outline new structure for experiments #9938

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b875770
Outline new structure for experiments
danielbachhuber Nov 21, 2024
c208295
Restore a few more
danielbachhuber Nov 21, 2024
8be2129
More edits
danielbachhuber Nov 21, 2024
c106de6
More edits
danielbachhuber Nov 21, 2024
f0e2941
More edits
danielbachhuber Nov 21, 2024
36ee205
More edits
danielbachhuber Nov 21, 2024
9ad7187
More
danielbachhuber Nov 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions contents/docs/experiments/new-outline.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: New outline (do not merge)
---

## Experiments

- Overview
- Keep as-is for now
- Installation
- Start here
- New doc with the key concepts you need to understand
- Outline:
Comment on lines +10 to +12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like where this is going. Curious what you think should go here?

Suggested change
- Start here
- New doc with the key concepts you need to understand
- Outline:
- Key concepts


## Common use-cases

- Click rate
- Trend with count
- Click-through rate
- Funnel
- Session length
- Trend with property value
- Retention ?

## Setting up

- [Creating an experiment](/docs/experiments/creating-an-experiment)
- Refresh doc based on new UI
- Outline:
- Navigate to Experiments and click "New experiment"
- Explanation of each form field:
- Name
- Feature flag key
- Description
- Participant type
- Variants
Comment on lines +29 to +35
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mostly the same, right?

- Holdout group
- Link to holdouts documentation
Comment on lines +36 to +37
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still keep holdouts as its own page. Easier to find

- Defining experiments with code
- Designing no-code experiments
- [Testing and launching](/docs/experiments/testing-and-launching)
- Keep the first part of the existing doc.
- Potentially rename to "Starting the experiment"
- Evaluating results
- Use the second part of the existing 'Testing and launching' doc.
Comment on lines +43 to +44
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good


## Features

- [Experiments without feature flags](/docs/experiments/running-experiments-without-feature-flags)
- Should this move to Guides?
Comment on lines +48 to +49
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the motivation behind moving it?

- [Holdouts](/docs/experiments/holdouts)
- Keep existing doc

## Methodology

- Statistics primer
- Trends with count
- Trends with property value
- Funnels
- Attribution type: Funnels allow you to have the attribution assigned at any point in the funnel, not just the first step. If the user assigned to a variant later on in the funnel, then they'll be included in the entire funnel.
- [Traffic allocation](/docs/experiments/traffic-allocation)
- [Sample size and running time](/docs/experiments/sample-size-running-time)
Comment on lines +53 to +61
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely sure what we talk about in these sections? But in general I think its a good direction

Suggested change
## Methodology
- Statistics primer
- Trends with count
- Trends with property value
- Funnels
- Attribution type: Funnels allow you to have the attribution assigned at any point in the funnel, not just the first step. If the user assigned to a variant later on in the funnel, then they'll be included in the entire funnel.
- [Traffic allocation](/docs/experiments/traffic-allocation)
- [Sample size and running time](/docs/experiments/sample-size-running-time)
## Under the hood
- Statistics primer
- Trends with count
- Trends with property value
- Funnels
- Attribution type: Funnels allow you to have the attribution assigned at any point in the funnel, not just the first step. If the user assigned to a variant later on in the funnel, then they'll be included in the entire funnel.
- [Traffic allocation](/docs/experiments/traffic-allocation)
- [Sample size and running time](/docs/experiments/sample-size-running-time)


## TBD

- [Experiment significance](/docs/experiments/experiment-significance)


- Chat bot that tells you how to set up an experiment?
Loading