Add ExPlat experiment: Payments task onboarding flow skips Connect page #9383
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Test the buildOption 1. Jetpack Beta
Option 2. Jurassic Ninja - available for logged-in A12s🚀 Launch a JN site with this branch 🚀 ℹ️ Install this Tampermonkey script to get more options. Build info:
Note: the build is updated when a new commit is pushed to this PR. |
|
Size Change: +5 B (0%) Total Size: 1.33 MB
ℹ️ View Unchanged
|
dmallory42
approved these changes
Sep 5, 2024
There was a problem hiding this comment.
This is looking great! Tested and works exactly as expected. Nice work, @vladolaru!
…onboarding-flow-skips-connect-page
necronet
reviewed
Sep 6, 2024
aaronyan
reviewed
Sep 6, 2024
There was a problem hiding this comment.
LGTM!
abtandabt_v
We generally don't encourage keeping track of variations and analyzing the data outside of ExPlat, since it's difficult to do correctly and we don't want a12s making decisions with bad data.
Instead, I'd recommend adding metrics in ExPlat (PCYsg-Fq6-p2) and then adding those metrics to your experiment. You get the benefit of all of our data cleaning, health checks, automated analysis, and statistical tests to determine uncertainty.
As an example of how to misinterpret the data, I see that the variation is set to control for many reasons (assigned control, user does not give tracking consent, user is ineligible, if there was an error getting variation, etc). So when the variation is saved inabt_v, it is saving what the user experienced not what they were assigned or if the user was eligible or not.
…onboarding-flow-skips-connect-page
|
Thank you for reviewing this, @aaronyan!
You are probably right. I intended those Tracks props to be a safety insurance of sorts to potentially allow us to ascertain onboarded accounts beyond Tracks (like account Risk approval rates). But, let's keep it simple. I removed the experiment-related Tracks props. |
|
Thank you for reviewing this, @necronet!
That use of _GET params was only for tracking purposes (which is gone now). For behavior change, we rely on |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #9382
Changes proposed in this Pull Request
We introduce the changes needed to point merchants starting their WooPayments onboarding flow from the WooCommerce Payments task (labeled "Get paid with WooPayments") directly to the Jetpack connection flow and/or WooPayments onboarding wizard (MOX), thus skipping the Connect page, if they are in the
treatmentvariation of the experiment.The experiment
controlgroup will continue going to the Connect page after clicking the "Get paid with WooPayments" task list item.Additionally, we include props about the active AB test and variation for certain onboarding Tracks events, for treatment users. Tracks events for control users will not have those props.(no longer the case)Testing instructions
Pre-testing setup
npm run build:clientUS)npm run tube:start) so you will be able to set up your Jetpack connectionTesting the control scenario
return false;line at the beginning of theWC_Payments_Utils::is_in_core_payments_task_onboarding_flow_treatment_mode()method to treat users as being in the control group.from=WCADMIN_PAYMENT_TASK:source=wcadmin-payment-task&from=WPCOM_CONNECTION:wcadmin_wcpay_onboarding_flow_startedevent with the following props (notice thesourceprop):wcadmin_wcpay_onboarding_flow_exitedevent with the following props (notice thesourceprop):from=WCADMIN_PAYMENT_TASK:wcadmin_wcpay_onboarding_flow_startedevent with the following props:wcadmin_wcpay_onboarding_flow_redirectedevent with the following props (notice thesourceprop):Testing the treatment scenario
return true;line at the beginning of theWC_Payments_Utils::is_in_core_payments_task_onboarding_flow_treatment_mode()method to treat users as being in the treatment group.source=wcadmin-payment-task&abt=cptof_2024_v1&abt_v=treatment&from=WPCOM_CONNECTION:wcadmin_wcpay_onboarding_flow_startedevent with the following props (notice thesourceprop):wcadmin_wcpay_onboarding_flow_exitedevent with the following props (notice thesourceprop):source=wcadmin-payment-task&abt=cptof_2024_v1&abt_v=treatment&from=WCADMIN_PAYMENT_TASKwcadmin_wcpay_onboarding_flow_startedevent with the following props (notice thesourceprop):wcadmin_wcpay_onboarding_flow_redirectedevent with the following props (notice thesourceprop):npm run changelogto add a changelog file, choosepatchto leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.Post merge