DOI matching in duplicate check #6897

Siedlerchr · 2020-09-09T14:35:49Z

Follow up from #6756

Change in CHANGELOG.md described (if applicable)
Tests created for changes (if applicable)
Manually tested changed features in running JabRef (always required)
Screenshots added in PR description (for UI changes)
Checked documentation: Is the information available and up to date? If not created an issue at https://github.com/JabRef/user-documentation/issues or, even better, submitted a pull request to the documentation repository.

Add zero weights for otes and comments.

small improvements

Moving cleanup's logic to DOI constructor.Add results of generate-authors.sh.

Moving cleanup's logic to DOI constructor.Add results of generate-authors.sh. (revert AUTHORS)

into KunAndrew-fix-for-issue-6707 * 'fix-for-issue-6707' of https://github.com/KunAndrew/jabref: issue 6707 Moving cleanup's logic to DOI constructor.Add results of generate-authors.sh. (revert AUTHORS) issue 6707 Moving cleanup's logic to DOI constructor.Add results of generate-authors.sh. issue 6707 small improvements issue 6707 Add zero weights for otes and comments. Add DOI matching with not full coincidence.

koppor · 2020-09-09T14:36:43Z

The JabRef maintainers will add the following name to the AUTHORS file. In case you want to use a different one, please comment here and adjust your name in your git configuration for future commits.

Andrew
Andrew Kuncevich

Siedlerchr · 2020-09-09T14:43:15Z

@KunAndrew Your problems probably came from the fact that your repo's master branch is behind the naster branch of jabref.

on the commandlne check that git remote -v has listed sth like upstream git@github.com:JabRef/jabref.git (can also the https link)
If not then do:git remote add upstream git@github.com:JabRef/jabref.git(if you use ssh otherwise use the https link)
git checkout master
git merge upstream/master
Then you master branch should be even with JabRef's upstream branch, so creating a new branch and creating a PR won't poroduce any conflicts

Siedlerchr · 2020-09-10T05:48:50Z

src/main/java/org/jabref/model/entry/identifier/DOI.java

@@ -108,12 +108,12 @@ public DOI(String doi) {
        Matcher matcher = EXACT_DOI_PATT.matcher(trimmedDoi);
        if (matcher.find()) {
            // match only group \1
-            this.doi = matcher.group(1);
+            this.doi = matcher.group(1).replaceAll("[^\\w,/,:,-,.,-]|[_]", "");


@KunAndrew I think you need to adjust the DOI regex pattern:
One CrossRef test fails, because of the missing underscore:
The DOI cannot be resolved wihtout underscore
org.opentest4j.AssertionFailedError: expected: <10.1007/11538394_20> but was: <10.1007/1153839420>

There should be a comment added which characters are removed and why. Without any comment, this is a "magic number" which should be extraced to a constant.

I removed this addtional regex because it was not really making sense. The DOI class already trims DOIs and also removes other encoding components.

src/main/java/org/jabref/logic/database/DuplicateCheck.java

koppor

Small code comments

src/main/java/org/jabref/logic/database/DuplicateCheck.java

koppor · 2020-09-10T19:10:13Z

src/main/java/org/jabref/model/entry/identifier/DOI.java

@@ -108,12 +108,12 @@ public DOI(String doi) {
        Matcher matcher = EXACT_DOI_PATT.matcher(trimmedDoi);
        if (matcher.find()) {
            // match only group \1
-            this.doi = matcher.group(1);
+            this.doi = matcher.group(1).replaceAll("[^\\w,/,:,-,.,-]|[_]", "");


There should be a comment added which characters are removed and why. Without any comment, this is a "magic number" which should be extraced to a constant.

src/main/java/org/jabref/model/entry/identifier/DOI.java

Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>

…-issue-6707 * upstream/master: (35 commits) Fix a fetcher test for the ShortDOIService (#6945) Fixes Shared Database: Changes filtering in CoarseChangeFilter to attribute property (#6868) Changed default value of "search and store files relative to bibtex file" to true (#6928) Replace comment by just a failure (#6943) Fix: in entry types editor selected field is not removed after first click (#6941) Fix remove actions for entry types in the editor (#6933) Always use Java 15 (#6929) Update DevDocs: workaround for issues with local openjfx maven libraries (#6931) Fixes bugs in the `regex` cite key pattern modifier (#6893) Add missing author Readability for citation key patterns (#6706) Add new author Reset to master and add default case to switch (#6847) Bump mockito-core from 3.5.10 to 3.5.11 (#6924) Bump byte-buddy-parent from 1.10.14 to 1.10.15 (#6923) Bump org.beryx.jlink from 2.21.4 to 2.22.0 (#6925) Bump xmpbox from 2.0.20 to 2.0.21 (#6926) Bump pascalgn/automerge-action from v0.9.0 to v0.10.0 (#6927) Improve parsing of short DOIs (#6920) Bump junit-vintage-engine from 5.6.2 to 5.7.0 (#6910) ...

remove additional regex we already are trimming and normalizing DOIs

KunAndrew and others added 6 commits August 8, 2020 16:36

Add DOI matching with not full coincidence.

6c8b7d7

issue 6707

25e34f0

Add zero weights for otes and comments.

issue 6707

8e4714b

small improvements

issue 6707

b9f412c

Moving cleanup's logic to DOI constructor.Add results of generate-authors.sh.

issue 6707

8a02474

Moving cleanup's logic to DOI constructor.Add results of generate-authors.sh. (revert AUTHORS)

Siedlerchr added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Sep 9, 2020

remove unnecessary cast

3fc1ffa

Siedlerchr commented Sep 10, 2020

View reviewed changes

tobiasdiez added status: changes required Pull requests that are not yet complete and removed status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers labels Sep 10, 2020

koppor reviewed Sep 10, 2020

View reviewed changes

src/main/java/org/jabref/logic/database/DuplicateCheck.java Outdated Show resolved Hide resolved

koppor requested changes Sep 10, 2020

View reviewed changes

koppor reviewed Sep 15, 2020

View reviewed changes

src/main/java/org/jabref/model/entry/identifier/DOI.java Outdated Show resolved Hide resolved

Siedlerchr and others added 5 commits September 26, 2020 13:45

Update src/main/java/org/jabref/logic/database/DuplicateCheck.java

d8c08a2

Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>

Update src/main/java/org/jabref/logic/database/DuplicateCheck.java

0aee547

Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>

Update src/main/java/org/jabref/model/entry/identifier/DOI.java

074856d

Co-authored-by: Oliver Kopp <kopp.dev@gmail.com>

simplify code

0fa5691

remove additional regex we already are trimming and normalizing DOIs

Siedlerchr added status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers and removed status: changes required Pull requests that are not yet complete labels Sep 26, 2020

Siedlerchr added 2 commits September 26, 2020 14:39

add changelog entry

298ec85

Merge branch 'master' into KunAndrew-fix-for-issue-6707

3920cd3

calixtus approved these changes Sep 27, 2020

View reviewed changes

koppor approved these changes Sep 29, 2020

View reviewed changes

koppor merged commit 8a242c9 into master Sep 29, 2020

koppor deleted the KunAndrew-fix-for-issue-6707 branch September 29, 2020 06:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOI matching in duplicate check #6897

DOI matching in duplicate check #6897

Siedlerchr commented Sep 9, 2020 •

edited

Loading

koppor commented Sep 9, 2020

Siedlerchr commented Sep 9, 2020

Siedlerchr Sep 10, 2020 •

edited

Loading

koppor Sep 10, 2020

Siedlerchr Sep 26, 2020

koppor left a comment

koppor Sep 10, 2020

DOI matching in duplicate check #6897

DOI matching in duplicate check #6897

Conversation

Siedlerchr commented Sep 9, 2020 • edited Loading

koppor commented Sep 9, 2020

Siedlerchr commented Sep 9, 2020

Siedlerchr Sep 10, 2020 • edited Loading

Choose a reason for hiding this comment

koppor Sep 10, 2020

Choose a reason for hiding this comment

Siedlerchr Sep 26, 2020

Choose a reason for hiding this comment

koppor left a comment

Choose a reason for hiding this comment

koppor Sep 10, 2020

Choose a reason for hiding this comment

Siedlerchr commented Sep 9, 2020 •

edited

Loading

Siedlerchr Sep 10, 2020 •

edited

Loading