Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev/core#720 Remove median & mode stats from contribution summary in order to improve performance #13630

Merged

Conversation

eileenmcnaughton
Copy link
Contributor

@eileenmcnaughton eileenmcnaughton commented Feb 18, 2019

Overview

Remove mode & median from stats on contribution search summary and contribution tab on contacts as they take many times longer to render than the rest of the search. This causes a slow user experience on small to medium size sites (up to 100k contributions) and serious lags on larger sites and it was agree that the value of these stats does not warrant the pain caused

https://lab.civicrm.org/dev/core/issues/720

Before

Stats present, slow to render

After

Stats gone - page renders much faster

Technical Details

Full discussion at https://lab.civicrm.org/dev/core/issues/720 - including why the queries cannot be made fast as the annual queries on the contribution summary were

I also move the functions used to calculate the stats over to the one report which calls them. Any custom reports that still call them won't get a fatal error but the stats will not render

Comments

@civibot
Copy link

civibot bot commented Feb 18, 2019

(Standard links)

@civibot civibot bot added the master label Feb 18, 2019
@eileenmcnaughton eileenmcnaughton changed the title dev/core#720 Remove median & mode stats from contribution summary in order to impr… dev/core#720 Remove median & mode stats from contribution summary in order to improve performance Feb 18, 2019
@eileenmcnaughton
Copy link
Contributor Author

@colemanw here it is - this is going to be so good for people's impressions of how fast Civi is.

@@ -46,14 +46,10 @@
<th class="left contriTotalRight"> &nbsp; {ts}# Completed{/ts} &ndash; {$contributionSummary.total.count}</th>
</tr><tr>
<th class="contriTotalLeft">{ts}Avg{/ts} &ndash; {$contributionSummary.total.avg}</th>
<th class="right"> &nbsp; {ts}Median{/ts} &ndash; {$contributionSummary.total.median}</th>
<th class="right contriTotalRight"> &nbsp; {ts}Mode{/ts} &ndash; {$contributionSummary.total.mode}</th>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you have to add that dumb contriTotalRight class to the only th left in this row, so it will have both contriTotalRight and contriTotalLeft classes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please do this one too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colemanw can you test something locally - I get the best results for a contact with more than one currency by removing the tpl handling for more than one currency ie

          {if $contributionSummary.total.amount}
            <th class="contriTotalLeft right">{ts}Total{/ts} &ndash; {$contributionSummary.total.amount}</th>
            <th class="right"> &nbsp; {ts}# Completed{/ts} &ndash; {$contributionSummary.total.count}</th>
            <th class="right contriTotalRight"> &nbsp; {ts}Avg{/ts} &ndash; {$contributionSummary.total.avg}</th>
          {/if}

and NO

{if $contributionSummary.total.currencyCount gt 1}

Might be something we can just strip out?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

screenshot 2019-02-18 14 08 17

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colmanw - I just pushed that up

{else}
<th class="contriTotalLeft right">{ts}Total{/ts} &ndash; {$contributionSummary.total.amount}</th>
<th class="right"> &nbsp; {ts}# Completed{/ts} &ndash; {$contributionSummary.total.count}</th>
<th class="right"> &nbsp; {ts}Avg{/ts} &ndash; {$contributionSummary.total.avg}</th>
<th class="right"> &nbsp; {ts}Median{/ts} &ndash; {$contributionSummary.total.median}</th>
<th class="right contriTotalRight"> &nbsp; {ts}Mode{/ts} &ndash; {$contributionSummary.total.mode}</th>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you have to add that dumb contriTotalRight class to the last th in this row.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colemanw do you think I should put it back or should we do something else once merged?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go ahead and add the class to the 3rd th for now. We can kill those dumb classes in a followup

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - does that look better?

@eileenmcnaughton eileenmcnaughton force-pushed the goodbye_mode_and_median_hello_speed branch 2 times, most recently from 2d8edca to 1600345 Compare February 18, 2019 03:40
@eileenmcnaughton eileenmcnaughton force-pushed the goodbye_mode_and_median_hello_speed branch from 1600345 to bfac166 Compare February 18, 2019 05:16
@eileenmcnaughton
Copy link
Contributor Author

@colemanw this is all passing now

@colemanw
Copy link
Member

  • General standards
    • (r-explain) Pass
    • (r-user) Pass: Most users don't care about these stats. Those that do can find them in CiviReport.
    • (r-doc) Undecided - I think it should be documented somewhere how to get these stats from CiviReport.
    • (r-run) Pass: Works on the demo site.
  • Developer standards

@colemanw colemanw merged commit 4599a82 into civicrm:master Feb 19, 2019
@colemanw colemanw deleted the goodbye_mode_and_median_hello_speed branch February 19, 2019 01:36
@eileenmcnaughton
Copy link
Contributor Author

Thanks @colemanw
FWIW I did some checks - loading a contact with around 20k contributions took 33 seconds before & now 9 - it was over a minute before I started on this round of changes

I search for 'all queries of financial type x in January' went from 1.5 minutes to 10 seconds - in this case I did it on our staging server which has no data for January so that extra 1 min 20 seconds was on a no-results query

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants