Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(REF; dev/core#873) MailingAB - Migrate "copy winner" logic from JS to PHP #14045

Merged
merged 2 commits into from
Apr 25, 2019

Conversation

totten
Copy link
Member

@totten totten commented Apr 13, 2019

Overview

In CiviMail, a user may run an A/B test, compare the performance of each variant, and choose a "winner". The mailing details (subject, body_text, etc) are copied from the winner to the full, final mailing.

This refactoring does not seek to change the usage or behavior. It just moves the "copy winner" logic from in-situ JS to a PHP API. It aims to allow better unit-testing and to allow more alternative workflows.

Before

  • After the user chooses a winner, the details are copied via JS (crmMailingMgr.mergeInto(finalMailing, winnerMailing...)).

After

  • After the user chooses a winner, the details are copied via PHP API (MailingAB.submit winner_id=...)
  • Side benefit: Slightly reduces the number of AJAX calls. (Since the mailing details aren't changed on client-side, we don't need to send a Mailing.create to save any changes. But the client-side still winds up with the right data because it reloads after submission.)

Comments

The new test-coverage is for the new API parameter. To show that it has the overall intended effect (e.g. old code and new code produce similar outcomes for user), I used this procedure:

  • Make a test site
  • Setup a fake mail service (e.g. mailcatcher or fakesmtp)
  • Add all the sample contacts to the newsletter group
  • Compose a new A/B test (based on full email content -- different subject and different body). Target the newsletter group. Run Job.process_mailings. Check that a subset of people receive both mailings.
  • Save a DB snapshot
  • In different configurations (with patch; without patch; with "A" as winner; with "B" as winner; etc), do these steps and compare outcome:
    • Restore DB snapshot
    • Navigate to review A/B test results
    • Choose a winner (A or B)
    • Observe that the web UI refreshes (filling the winner/C column)
    • Run Job.process_mailings
    • Check that the remaining people receive the designated subject+body.

totten added 2 commits April 13, 2019 12:01
When a "winner" is selected for an A/B test, several fields must be copied over.
This provides the "copy" logic on the server-side.

Before
-------

The copy logic is only available in Javascript, so it's hard to reference in a unit-test.

After
-----

The copy logic is available as an extra option for the MailingAB.submit API.
By default, the API works as before.  If you add the option `winner_id`,
then it will also copy-over the details from the winning experiment.
…ilingMgr.mergeInto`

In this context, the user has run an A/B experiment and chosen a specific
variant (the "winner") to use in the full blast.  The purpose of the code is
to copy the "winner" content to the final mailing - and send the final
mailing.

This replaces some client-side logic with a server-side equivalent.
@civibot
Copy link

civibot bot commented Apr 13, 2019

(Standard links)

@eileenmcnaughton
Copy link
Contributor

@seamuslee001 are you in a position to test this? The bulk of the change is unit tests :-)

@eileenmcnaughton
Copy link
Contributor

Code makes sense & unit tests are solid - I understand you have work building on this. I discussed with @seamuslee001 who also looked through the code. Merging

@eileenmcnaughton eileenmcnaughton merged commit f6b2492 into civicrm:master Apr 25, 2019
@totten totten deleted the master-ab-winning branch June 5, 2019 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants