Add PersianStemmer [LUCENE-10312]

[asfimport]

---

Migrated from LUCENE-10312 by Ramin Alirezaee, resolved May 19 2022
Attachments: image.png
Linked issues:

• Reduce class/member visibility of all normalizer and stemmer classes [LUCENE-10561] #11597

Pull requests: #540

[asfimport]

ASF subversion and git services (migrated from JIRA)

Commit 111d6b1 in lucene's branch refs/heads/main from Ramin ALirezaee
https://gitbox.apache.org/repos/asf?p=lucene.git;h=111d6b186e7

LUCENE-10312: Add PersianStemmer (#540)

Co-authored-by: Tomoko Uchida <tomoko.uchida.1111@gmail.com>

[asfimport]

ASF subversion and git services (migrated from JIRA)

Commit 9f04771 in lucene's branch refs/heads/branch_9x from Ramin ALirezaee
https://gitbox.apache.org/repos/asf?p=lucene.git;h=9f04771f311

LUCENE-10312: Add PersianStemmer (#540)

Co-authored-by: Tomoko Uchida <tomoko.uchida.1111@gmail.com>

[asfimport]

Alan Woodward (@romseygeek) (migrated from JIRA)

Hi, it looks like this adds the new PersianStemmer to all PersianAnalyzer instances, but that will cause compatibility issues as somebody who indexed using a PersianAnalyzer in 9.1 may find that they don't get hits any more when searching using 9.2 because the results of their analysis chain would be different.  I think we need to add stemming as a configuration option that is disabled by default, so that you can opt in to the new stemmer but we don't break backwards compatibility.

[asfimport]

Tomoko Uchida (@mocobeta) (migrated from JIRA)

@romseygeek thanks for noticing this! I was careless when backporting.
We could make PersianAnalyzer configurable so that users can opt in the new stemmer though, I simply reverted the changes to the Analyzer from 9x branch (I'd assume users who have the knowledge to configure the off-the-shelf Analyzers can also easily create custom analyzers on their own).
#904

Would you please review it?

[asfimport]

Alan Woodward (@romseygeek) (migrated from JIRA)

No problem @mocobeta - I think we can add a simple boolean parameter to PersianAnalyzer to configure stemming.  It would be a shame to just drop this out of 9.2 entirely as it's a nice addition to our analysis module.  I'll raise a PR to add configurability.

[asfimport]

Robert Muir (@rmuir) (migrated from JIRA)

There need not be backwards compatibility issues: Just add the stemmer to PersianAnalyzer in 10.x only. That's what major releases are for.

You can add to MIGRATE.txt, if you really want, how to build a CustomAnalyzer that looks like the old one, for people that are stuck in the past.

Please, do not add crazy configuration options like this.

[asfimport]

ASF subversion and git services (migrated from JIRA)

Commit 212fd3a in lucene's branch refs/heads/branch_9x from Tomoko Uchida
https://gitbox.apache.org/repos/asf?p=lucene.git;h=212fd3a7482

LUCENE-10312: Revert changes in PersianAnalyzer from 9x (#904)

[asfimport]

ASF subversion and git services (migrated from JIRA)

Commit ba8c3a8 in lucene's branch refs/heads/branch_9_2 from Tomoko Uchida
https://gitbox.apache.org/repos/asf?p=lucene.git;h=ba8c3a806ad

LUCENE-10312: Revert changes in PersianAnalyzer from 9x (#904)

[asfimport]

Tomoko Uchida (@mocobeta) (migrated from JIRA)

We addressed the backward compatibility issue. I'm closing this to avoid possible confusion in the 9.2 release operation but will make some necessary follow-ups (MIGRATE entry, javadocs, minor refactoring on tests) on main.

[asfimport]

ASF subversion and git services (migrated from JIRA)

Commit 71a9acb in lucene's branch refs/heads/main from Tomoko Uchida
https://gitbox.apache.org/repos/asf?p=lucene.git;h=71a9acb2e2a

LUCENE-10312: MIGRATE entry and small follow-ups (#908)

[asfimport]

Alan Woodward (@romseygeek) (migrated from JIRA)

Bulk close for 9.2.0 release