feat(experiments): add p-value #28098
Conversation
There was a problem hiding this comment.
PR Summary
Added p-value column to experiment results, providing statistical significance information for variant comparisons.
- Added new
pValuecolumn in/frontend/src/scenes/experiments/ExperimentView/SummaryTable.tsxthat displays 1 - probability with special formatting for values < 0.001 - Added
EXPERIMENT_P_VALUEfeature flag in/frontend/src/lib/constants.tsxto control p-value visibility - Column positioned before win probability, maintaining consistent table styling and handling undefined cases
- P-value calculation helps assess statistical significance of experiment results with 95% confidence level
2 file(s) reviewed, 3 comment(s)
Edit PR Review Bot Settings | Greptile
| const pValue = | ||
| result?.probability?.[variantKey] !== undefined ? 1 - result.probability[variantKey] : undefined |
There was a problem hiding this comment.
logic: The p-value calculation (1 - probability) assumes probability represents the complement of the p-value. Verify this assumption with the backend implementation.
| {pValue != undefined ? ( | ||
| <span className="inline-flex items-center w-52 space-x-4"> |
There was a problem hiding this comment.
style: Use strict equality (!==) instead of loose equality (!=) for undefined check to maintain consistency with TypeScript best practices
| {pValue != undefined ? ( | |
| <span className="inline-flex items-center w-52 space-x-4"> | |
| {pValue !== undefined ? ( | |
| <span className="inline-flex items-center w-52 space-x-4"> |
| @@ -239,6 +239,7 @@ export const FEATURE_FLAGS = { | |||
| LLM_OBSERVABILITY: 'llm-observability', // owner: #team-ai-product-manager | |||
| ONBOARDING_SESSION_REPLAY_SEPERATE_STEP: 'onboarding-session-replay-separate-step', // owner: @joshsny #team-growth | |||
There was a problem hiding this comment.
syntax: 'SEPERATE' is misspelled in the feature flag name above - should be 'SEPARATE'
| ONBOARDING_SESSION_REPLAY_SEPERATE_STEP: 'onboarding-session-replay-separate-step', // owner: @joshsny #team-growth | |
| ONBOARDING_SESSION_REPLAY_SEPARATE_STEP: 'onboarding-session-replay-separate-step', // owner: @joshsny #team-growth |
|
Size Change: +40 B (0%) Total Size: 1.17 MB ℹ️ View Unchanged
|
📸 UI snapshots have been updated1 snapshot changes in total. 0 added, 1 modified, 0 deleted:
Triggered by this commit. |
📸 UI snapshots have been updated1 snapshot changes in total. 0 added, 1 modified, 0 deleted:
Triggered by this commit. |
📸 UI snapshots have been updated1 snapshot changes in total. 0 added, 1 modified, 0 deleted:
Triggered by this commit. |
Changes
Feature flag:
experiment-p-valueHow did you test this code?
👀