Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dev/core#1230 [Dedupe] Do not geocode while merging, rely on existing values #15154

Merged
merged 1 commit into from
Sep 6, 2019

Conversation

eileenmcnaughton
Copy link
Contributor

@eileenmcnaughton eileenmcnaughton commented Aug 29, 2019

Overview

This is a performance fix when merging - don't do geocoding when merging - we already have
geocode data for the contacts. We also move the 'whole address' so shouldn't re-geocode for more info

https://issues.civicrm.org/jira/browse/CRM-21786

Before

Geocoding calls happen during merging

After

No calls (existing geocode info in use)

Technical Details

Comments

@pfigel @lcdservices

This is a performance fix when merging - on a batch merge job it can be kinda crazy & we already have
geocode data for the contacts
@civibot
Copy link

civibot bot commented Aug 29, 2019

(Standard links)

@civibot civibot bot added the master label Aug 29, 2019
@eileenmcnaughton
Copy link
Contributor Author

test this please

@eileenmcnaughton
Copy link
Contributor Author

test this please

1 similar comment
@eileenmcnaughton
Copy link
Contributor Author

test this please

@eileenmcnaughton eileenmcnaughton changed the title Do not geocode while merging, rely on existing values [Dedupe] Do not geocode while merging, rely on existing values Sep 3, 2019
@eileenmcnaughton eileenmcnaughton changed the title [Dedupe] Do not geocode while merging, rely on existing values dev/core#1230 [Dedupe] Do not geocode while merging, rely on existing values Sep 4, 2019
@demeritcowboy
Copy link
Contributor

I can do an r-run just an initial comment: I'm not familiar with where else the skip parameter can be set or if this code can be run outside the UI but even if it's not possible right now for the skip parameter to have been previously set and even if CRM_Utils_Address::format() and anything it might call doesn't use it currently and even though there is no other code in that block currently, it might be more future proof to save the old value and replace after?

$old_skip_geocode = NULL;
if (isset($value['skip_geocode'])) {
  $old_skip_geocode = $value['skip_geocode'];
}
$value['skip_geocode'] = TRUE;
CRM_Core_BAO_Address::fixAddress($value);
if (isset($old_skip_geocode)) {
  $value['skip_geocode'] = $old_skip_geocode;
} else {
  unset($value['skip_geocode']);
}

Also to address the other comment clearly fixAddress() fixes the address (grin). There's even a comment in there explaining that because of "network latency" sometimes non-numeric string values manage to get in there so they need to be ignored... Ahh good old 2007.

@demeritcowboy
Copy link
Contributor

  • General standards
    • [r-explain] PASS
    • [r-user] PASS
    • [r-doc] PASS
    • [r-run] PASS
      • I didn't do a before and after speed test on a lot of contacts but as to whether it seems to all work it looks good.
      • There's a situation I did test where if the winner contact has an ungeocoded address and the loser contact has the same address but it is geocoded and you choose not to overwrite (because they seem the same on the merge screen and the default is don't overwrite) then the winning result isn't geocoded either. But that seems picky and is easily manually or batch-update fixable.
  • Developer standards
    • [r-tech] PASS
    • [r-code] PASS
    • [r-maint] N/A
    • [r-test] PASS

@eileenmcnaughton
Copy link
Contributor Author

Merging per review.

In response to comment one $value is really a throw-away parameter so resetting it is really just for clarity

On the edge case - I think if some addresses are not geocoding them that is a system issue & we shouldn't slow down deduping for it

@eileenmcnaughton eileenmcnaughton merged commit 46880a3 into civicrm:master Sep 6, 2019
@eileenmcnaughton eileenmcnaughton deleted the dedupe3 branch September 6, 2019 00:59
@demeritcowboy
Copy link
Contributor

It's currently a throwaway parameter, but in the future there might be more code in that block, or format() might end up using it or passing it on to something that does. But I don't see it as a blocker.

@eileenmcnaughton
Copy link
Contributor Author

Please please we want less not more code there....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants