Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(dev/core#217) PrevNext - More conservative transition (5.7=>5.8=>5.9) #13164

Merged
merged 5 commits into from
Nov 28, 2018

Conversation

totten
Copy link
Member

@totten totten commented Nov 28, 2018

Overview

This revision sets up more conservative transition process for adopting new PrevNext drivers.

Before

  • 5.7.x - Hard-coded to SQL driver.
  • 5.8.x - Allow admin use a setting to pick a driver. If left as default, then auto-detect best-available (based on configured services).

After

  • 5.7.x - Hard-coded to SQL driver.
  • 5.8.x - Allow admin use a setting to pick a driver. If left as default, then use SQL driver.
  • 5.9.x - Allow admin use a setting to pick a driver. If left as default, then auto-detect best-available (based on configured services).

This essentially mitigates the risk that bugs in the new Redis driver cause regreessions for sites already running Redis.

Comments

This extends/depends on #12665. The new change here is only one commit - the last one, 318583d.

An alternative way to achieve the same goal (more conservative transition) would be to default to SQL under all circumstances -- and (if Redis is available, then...) show a status message suggesting that the admin change the settings. OTOH, 6 months from now, we'll probably be happier to just have it auto-detect the best driver. I'm OK to close and do a different PR if folks really want that process instead, but either approach seems fair enough IMHO.

…ad of SQL JOIN

The general context of this code is roughly as follows:

* We've already filled up the prevnext cache with a bunch of contact-IDs.
* The user wants to view a page of 50 contacts.
* We want to lookup full information about 50 specific contacts for this page.

It does makes sense to use `CRM_Contact_BAO_Query` for looking up the "full information"
about contacts. However, the function `Query::getCachedContacts()` is hard-coded to
read from the SQL-based prevnext cache.

Before
------

* In `getCachedContacts()`, it grabbed the full SQL for `CRM_Contact_BAO_Query`
  and munged the query to:
    * Add an extra JOIN on `civicrm_prevnext_cache` (with a constraint on `cacheKey`)
    * Respect pagination (LIMIT/OFFSET)
    * Order results based on their position in the prevnext cache

After
-----

* In `CRM_Core_PrevNextCache_Interface`, the `fetch()` function provides one page-worth
  of contact IDs (in order). The `fetch()` function is tested by `E2E_Core_PrevNextTest`.
* In `getCachedContacts()`, it doesn't know anything about `civicrm_prevnext_cache`
  or `cacheKey` or pagination.  Instead, it just accepts CIDs for one page-worth of
  contacts.  It returns contacts in the same order that was given.
The new name is prettier and matches the names in `CRM_Core_PrevNextCache_{Interface,Redis}`.
The auto-detection is a good default policy.  However, this is new
functionality.  If some bug gets through the review/RC cycles, then this
option provides an escape path.
This revision sets up more conservative transition process for adopting PrevNext drivers.

Before
------

* 5.7.x - Hard-coded to SQL driver.
* 5.8.x - Allow setting to pick driver. If left as `default`, then choose best-available (based on configured services).

After
------

* 5.7.x - Hard-coded to SQL driver.
* 5.8.x - Allow setting to pick driver. If left as `default`, then use SQL driver.
* 5.9.x - Allow setting to pick driver. If left as `default`, then choose best-available (based on configured services).

This essentially mitigates the risk that bugs in the new Redis driver cause regreessions for sites already running Redis.
@civibot
Copy link

civibot bot commented Nov 28, 2018

(Standard links)

@civibot civibot bot added the master label Nov 28, 2018
@totten
Copy link
Member Author

totten commented Nov 28, 2018

NOTE: Relevant comment from @eileenmcnaughton on 12665:

@totten so in fact we have been running in production still on m[y]sql caching - but I'm testing with Redis on staging. I'm just running some performance benchmarks at the moment - maybe we can get some Redis users to test the rc & we can add the conservative approach if need be? - I think @herbdool uses Redis

We can close or merge this PR whenever that feedback comes.

@eileenmcnaughton
Copy link
Contributor

Ah well no harm in being more conservative I guess. I can't see a downside in merging this so let's do it

@eileenmcnaughton
Copy link
Contributor

I'm gonna try 'rebase and merge' just to see - I suspect I'll lose the merge commit but we'll see

@eileenmcnaughton eileenmcnaughton merged commit f76de01 into civicrm:master Nov 28, 2018
@eileenmcnaughton
Copy link
Contributor

OK - I can confirm using that lost me some history - it makes it look like you pushed a commit directly. If anyone accuses you of that you can point the finger at me.

I will avoid using that in future

@totten totten deleted the master-prevnext-redis-trans branch November 28, 2018 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants