Merge branch 'main' into feature/binary_search_in_bkd

* main:
  LUCENE-10456: Implement Weight#count for MultiRangeQuery (apache#731)
  LUCENE-10484: Move CHANGES entry to 9.2.
  LUCENE-10484: Add support for concurrent facets random sampling (apache#765)
  LUCENE-10002: Replaces usages of FacetsCollector with FacetsCollectorManager (apache#764)
  LUCENE-10466: Move changelog entry to 9.2
  LUCENE-10466: Ensure IndexSortSortedNumericDocValuesRangeQuery handles sort types besides LONG
  LUCENE-10498: don't count num hits nor accumulate scores unless necessary (apache#782)
  LUCENE-10184: mention of opening a Jira issue (apache#781)
  Update link to contribution guide
  Remove redundant index (apache#776)
  Implement method to bulk add all collection elements to a PriorityQueue (apache#770)
  LUCENE-10491: Fix correctness bug in TaxonomyFacetSumValueSource score providing (apache#775)
  Replace usages of search(Query, Collector) in CheckHits (apache#763)
  LUCENE-10002: Add FixedBitSetCollector and corresponding collector manager to test framework (apache#766)
  LUCENE-10475: Merge o.a.l.a.[ja|ko].util into o.a.l.a.[ja|ko].dict (apache#772)
  LUCENE-10184: add CONTRIBUTING.md; reorganize README. (apache#771)
  Add CHANGES entry for LUCENE-10325
  LUCENE-10325: Add getTopDims functionality to Facets (apache#747)

.github/PULL_REQUEST_TEMPLATE.md

@@ -32,7 +32,7 @@ Please describe the tests you've developed or run to confirm this patch implemen
 
 Please review the following and check all that apply:
 
-- [ ] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/lucene/HowToContribute) and my code conforms to the standards described there to the best of my ability.
+- [ ] I have reviewed the guidelines for [How to Contribute](https://github.com/apache/lucene/blob/main/CONTRIBUTING.md) and my code conforms to the standards described there to the best of my ability.
 - [ ] I have created a Jira issue and added the issue ID to my pull request title.
 - [ ] I have given Lucene maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended)
 - [ ] I have developed this patch against the `main` branch.

CONTRIBUTING.md

@@ -0,0 +1,128 @@
+<!--
+    Licensed to the Apache Software Foundation (ASF) under one or more
+    contributor license agreements.  See the NOTICE file distributed with
+    this work for additional information regarding copyright ownership.
+    The ASF licenses this file to You under the Apache License, Version 2.0
+    the "License"); you may not use this file except in compliance with
+    the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing, software
+    distributed under the License is distributed on an "AS IS" BASIS,
+    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    See the License for the specific language governing permissions and
+    limitations under the License.
+ -->
+
+# Contributing to Lucene Guide
+
+## Working with Code
+
+### Getting the source code
+
+First of all, you need the Lucene source code.
+
+Get the source code using: `git clone https://github.com/apache/lucene`
+
+### Notes for required Java version
+
+Be sure that you are using an appropriate version of the JDK. Please check [README](./README.md) for the required JDK version for current main branch.
+
+### Building with Gradle
+
+Lucene uses [Gradle](https://gradle.org/) for build control. Gradle is itself Java-based and may be incompatible with newer Java versions; you can still build and test Lucene with these Java releases, see [jvms.txt](./help/jvms.txt) for more information.
+
+Run "./gradlew help", this will show the main tasks that can be executed to show help sub-topics.
+
+If you want to build Lucene, type:
+
+```
+./gradlew assemble
+```
+
+NOTE: DO NOT use the `gradle` command that is perhaps installed on your machine. This may result in using a different gradle version than the project requires and this is known to lead to very cryptic errors. The "gradle wrapper" (gradlew script) does everything required to build the project from scratch: it downloads the correct version of gradle, sets up sane local configurations and is tested on multiple environments.
+
+The first time you run gradlew, it will create a file "gradle.properties" that contains machine-specific settings. Normally you can use this file as-is, but it can be modified if necessary.
+
+`./gradlew check` will assemble Lucene and run all validation
+  tasks (including tests).
+
+`./gradlew help` will print a list of help guides that introduce and explain
+  various parts of the build system, including typical workflow tasks.
+
+If you want to just build the documentation, type:
+
+```
+./gradlew documentation
+```
+
+### Checks
+
+Please make sure that all unit tests and validations succeed before constructing your patch: `./gradlew check`.
+
+To run a single test case from the lucene/core directory in your working copy:  `./gradlew -p lucene/core test --tests NameOfYourUnitTest`. Run `./gradlew helpTests` to get more information about running tests.
+
+In case your contribution fixes a bug, please create a new test case that fails before your fix, to show the presence of the bug and ensure it never re-occurs. A test case showing the presence of a bug is also a good contribution by itself.
+
+### IDE support
+
+- *IntelliJ* - IntelliJ idea can import and build gradle-based projects out of the box.
+- *Eclipse*  - Basic support ([help/IDEs.txt](https://github.com/apache/lucene/blob/main/help/IDEs.txt#L7)).
+- *Netbeans* - Not tested.
+
+## Contributing your work
+
+You have two options to contribute your work: you can either create a patch and attach it to an issue on [Jira](https://issues.apache.org/jira/browse/LUCENE), or open a pull request at https://github.com/apache/lucene - whichever works best for you.
+
+Please be patient. Committers are busy people too. If no one responds to your patch after a few days, please make friendly reminders. Please incorporate others' suggestions into into your patch if you think they're reasonable. Finally, remember that even a patch that is not committed is useful to the community.
+
+### Opening a Jira issue
+
+You may want to open a Jira issue before opening a pull request or creating a patch. This is especially required when adding a [CHANGES](./lucene/CHANGES.txt) entry. If your contribution is a small one (such as fixing typo in documentation) this can be optional.
+
+### Opening a pull request
+
+Please refer to [GitHub's documentation](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests) for an explanation of how to create a pull request. The pull request title should start with the Jira issue number that your work is related to, this way your pull request will get automatically linked from the Jira issue.
+
+### Creating a patch
+
+Note that you do not need to create a patch if you already opened a pull request on GitHub.
+
+Before creating your patch, you may want to get 'main' up to date with the latest from upstream. This will help avoid the possibility of others finding merge conflicts when applying your patch. This can be done with git pull if main is the current branch.
+
+If your changes are in your git working tree (i.e. not committed to a branch), then do this:
+
+1. Check to see what files you have modified with: `git status`
+2. Add any new files with: `git add src/.../MyNewClass.java`
+3. In order to create a patch, just type `git diff HEAD > LUCENE-NNNN.patch`
+
+If your changes are all committed to the current (feature) branch: `git diff main... > LUCENE-NNNN.patch`
+
+If some of your changes are committed to the current (feature) branch but you have working-copy changes too: `git diff {{git merge-base main head > LUCENE-NNNN.patch}}`. This will save a diff into the LUCENE-NNNN.patch file. Read the patch file. Make sure it includes ONLY the modifications required to fix a single issue.
+
+Patches should be attached to an issue in Jira.
+
+To enable automatic validation of Jira patches, switch the issue's status to "Patch Available" by clicking on the "Enable Patch Review" button near the top of the page. This will enqueue an ASF Jenkins job PreCommit-LUCENE-Build to run various quality checks on the patch and post a validation report as a comment (by "Lucene/Solr QA") on the issue. Expect a delay of 12 hours or so before the patch validation job actually runs. Note that in order for a patch file to trigger validation, its name must conform to the naming rules outlined here: https://yetus.apache.org/documentation/in-progress/precommit-patchnames/
+
+## Stay involved
+
+Contributors should join the [Lucene mailing lists](https://lucene.apache.org/core/discussion.html). In particular, the commit list (to see changes as they are made), the dev list (to join discussions of changes) and the user list (to help others).
+
+Please keep discussions about Lucene on list so that everyone benefits. Emailing individual committers with questions about specific Lucene issues is discouraged. See http://people.apache.org/~hossman/#private_q.
+
+## Getting your feet wet: where to begin?
+
+New to Lucene? Want to find JIRA issues that you can work on without taking on the whole world?
+
+The Lucene/Solr developers use the "newdev" label to mark issues that developers new to Lucene might be interested in working on. The rough criteria used to make this selection are:
+
+- Nobody has done any work on the issue yet.
+- The issue is likely not controversial.
+- The issue is likely self-contained with limited scope.
+
+To see a list of open Lucene issues with the newdev label, look at this link [http://s.apache.org/newdevlucenesolr](https://issues.apache.org/jira/browse/SOLR-15591?jql=(project%3DLUCENE)%20AND%20resolution%3DUnresolved%20AND%20labels%3Dnewdev).
+
+## Developer tips
+
+For more contribution guidelines and tips, see [DeveloperTips](https://cwiki.apache.org/confluence/display/LUCENE/DeveloperTips).

README.md

@@ -35,84 +35,23 @@ comprehensive documentation, visit:
 - Developer Documentation: [dev-docs/](./dev-docs/)
 - Migration Guide: [lucene/MIGRATE.md](./lucene/MIGRATE.md)
 
-## Building with Gradle
+## Building
 
 ### Basic steps:
 
-  0. Install OpenJDK 17 (or greater).
-  1. Clone Lucene's git repository (or download the source distribution).
-  2. Run gradle launcher script (`gradlew`).
+1. Install OpenJDK 17 (or greater).
+2. Clone Lucene's git repository (or download the source distribution).
+3. Run gradle launcher script (`gradlew`).
 
-### Step 0) Set up your development environment (OpenJDK 17 or greater)
+We'll assume that you know how to get and set up the JDK - if you don't, then we suggest starting at https://jdk.java.net/ and learning more about Java, before returning to this README.
 
-We'll assume that you know how to get and set up the JDK - if you
-don't, then we suggest starting at https://jdk.java.net/ and learning
-more about Java, before returning to this README. Lucene runs with
-Java 17 or later. 
-
-Lucene uses [Gradle](https://gradle.org/) for build control. Gradle is itself Java-based
-and may be incompatible with newer Java versions; you can still build and test 
-Lucene with these Java releases, see [jvms.txt](./help/jvms.txt) for more information.
-
-NOTE: Lucene changed from Apache Ant to Gradle as of release 9.0. Prior releases
-still use Apache Ant.
-
-### Step 1) Checkout/Download Lucene source code
-
-You can clone the source code from GitHub:
-
-  https://github.com/apache/lucene
-
-or get Lucene source archives for a particular release from:
-
-  https://lucene.apache.org/core/downloads.html
-
-Download the source archive and uncompress it into a directory of your choice.
-
-### Step 2) Run Gradle
-
-Run "./gradlew help", this will show the main tasks that can be executed to 
-show help sub-topics.
-
-If you want to build Lucene, type:
-
-```
-./gradlew assemble
-```
-
-NOTE: DO NOT use the `gradle` command that is perhaps installed on your machine. This may
-result in using a different gradle version than the project requires and this is known
-to lead to very cryptic errors. The "gradle wrapper" (gradlew script) does everything
-required to build the project from scratch: it downloads the correct version of gradle,
-sets up sane local configurations and is tested on multiple environments.
-
-The first time you run gradlew, it will create a file "gradle.properties" that
-contains machine-specific settings. Normally you can use this file as-is, but it
-can be modified if necessary.
-
-`./gradlew check` will assemble Lucene and run all validation
-  tasks (including tests).
-
-`./gradlew help` will print a list of help guides that introduce and explain
-  various parts of the build system, including typical workflow tasks.
-
-If you want to just build the documentation, type:
-
-```
-./gradlew documentation
-```
-
-### IDE support
-
-- *IntelliJ* - IntelliJ idea can import and build gradle-based projects out of the box.
-- *Eclipse*  - Basic support ([help/IDEs.txt](https://github.com/apache/lucene/blob/main/help/IDEs.txt#L7)).
-- *Netbeans* - Not tested.
+See [Contributing Guide](./CONTRIBUTING.md) for details.
 
 ## Contributing
 
 Bug fixes, improvements and new features are always welcome!
 Please review the [Contributing to Lucene
-Guide](https://cwiki.apache.org/confluence/display/lucene/HowToContribute) for information on
+Guide](./CONTRIBUTING.md) for information on
 contributing.
 
 ## Discussion and Support

gradle/generation/kuromoji.gradle

@@ -19,7 +19,7 @@
 
 def recompileDictionary(project, dictionaryName, Closure closure) {
   project.javaexec {
-    main = "org.apache.lucene.analysis.ja.util.DictionaryBuilder"
+    main = "org.apache.lucene.analysis.ja.dict.DictionaryBuilder"
     classpath = project.sourceSets.main.runtimeClasspath
 
     jvmArgs '-Xmx1G'

gradle/generation/nori.gradle

@@ -19,7 +19,7 @@
 
 def recompileDictionary(project, dictionaryName, Closure closure) {
   project.javaexec {
-    main = "org.apache.lucene.analysis.ko.util.DictionaryBuilder"
+    main = "org.apache.lucene.analysis.ko.dict.DictionaryBuilder"
     classpath = project.sourceSets.main.runtimeClasspath
 
     jvmArgs '-Xmx1G'

lucene/CHANGES.txt

@@ -39,6 +39,7 @@ Optimizations
 
 Bug Fixes
 ---------------------
+
 (No changes)
 
 Other
@@ -52,10 +53,15 @@ Other
 * LUCENE-10393: Unify binary dictionary and dictionary writer in Kuromoji and Nori.
   (Tomoko Uchida, Robert Muir)
 
+* LUCENE-10475: Merge dictionary builders in `util` package into `dict` package in Kuromoji and Nori.
+  All classes in `org.apache.lucene.analysis.[ja|ko].util` was moved to `org.apache.lucene.analysis.[ja|ko].dict`.
+  (Tomoko Uchida)
+
 ======================= Lucene 9.2.0 =======================
 API Changes
 ---------------------
-(No changes)
+
+* LUCENE-10325: Facets API extended to support getTopFacets. (Yuting Gan)
 
 New Features
 ---------------------
@@ -67,9 +73,17 @@ New Features
 implementation. `Monitor` can be created with a readonly `QueryIndex` in order to 
 have readonly `Monitor` instances. (Niko Usai)
 
+* LUCENE-10456: Implement rewrite and Weight#count for MultiRangeQuery
+by merging overlapping ranges . (Jianping Weng)
+
 Improvements
 ---------------------
-(No changes)
+
+* LUCENE-10494: Implement method to bulk add all collection elements to a PriorityQueue.
+  (Bauyrzhan Sakhariyev)
+
+* LUCENE-10484: Add support for concurrent random sampling by calling
+  RandomSamplingFacetsCollector#createManager. (Luca Cavanna)
 
 Optimizations
 ---------------------
@@ -88,6 +102,12 @@ Bug Fixes
 * LUCENE-10477: Highlighter: WeightedSpanTermExtractor.extractWeightedSpanTerms to Query#rewrite
   multiple times if necessary. (Christine Poerschke, Adrien Grand)
 
+* LUCENE-10491: A correctness bug in the way scores are provided within TaxonomyFacetSumValueSource
+  was fixed. (Michael McCandless, Greg Miller)
+
+* LUCENE-10466: Ensure IndexSortSortedNumericDocValuesRangeQuery handles sort field
+  types besides LONG (Andriy Redko)
+
 Build
 ---------------------
 

lucene/analysis/kuromoji/src/java/module-info.java

@@ -24,7 +24,6 @@
   exports org.apache.lucene.analysis.ja.completion;
   exports org.apache.lucene.analysis.ja.dict;
   exports org.apache.lucene.analysis.ja.tokenattributes;
-  exports org.apache.lucene.analysis.ja.util;
 
   opens org.apache.lucene.analysis.ja to
       org.apache.lucene.core;

lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/JapaneseReadingFormFilter.java

@@ -19,8 +19,8 @@
 import java.io.IOException;
 import org.apache.lucene.analysis.TokenFilter;
 import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.ja.dict.ToStringUtil;
 import org.apache.lucene.analysis.ja.tokenattributes.ReadingAttribute;
-import org.apache.lucene.analysis.ja.util.ToStringUtil;
 import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
 
 /**

lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/JapaneseTokenizer.java

@@ -265,7 +265,7 @@ public JapaneseTokenizer(
    * Create a new JapaneseTokenizer, supplying a custom system dictionary and unknown dictionary.
    * This constructor provides an entry point for users that want to construct custom language
    * models that can be used as input to {@link
-   * org.apache.lucene.analysis.ja.util.DictionaryBuilder}.
+   * org.apache.lucene.analysis.ja.dict.DictionaryBuilder}.
    *
    * @param factory the AttributeFactory to use
    * @param systemDictionary a custom known token dictionary

lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/util/CSVUtil.java → lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/CSVUtil.java

@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-package org.apache.lucene.analysis.ja.util;
+package org.apache.lucene.analysis.ja.dict;
 
 import java.util.ArrayList;
 import java.util.regex.Matcher;

lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/util/ConnectionCostsBuilder.java → lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/ConnectionCostsBuilder.java

@@ -14,15 +14,14 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-package org.apache.lucene.analysis.ja.util;
+package org.apache.lucene.analysis.ja.dict;
 
 import java.io.IOException;
 import java.io.LineNumberReader;
 import java.io.Reader;
 import java.nio.charset.StandardCharsets;
 import java.nio.file.Files;
 import java.nio.file.Path;
-import org.apache.lucene.analysis.ja.dict.ConnectionCosts;
 import org.apache.lucene.analysis.morph.ConnectionCostsWriter;
 
 class ConnectionCostsBuilder {

lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/util/DictionaryBuilder.java → lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/DictionaryBuilder.java

@@ -14,13 +14,12 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-package org.apache.lucene.analysis.ja.util;
+package org.apache.lucene.analysis.ja.dict;
 
 import java.io.IOException;
 import java.nio.file.Path;
 import java.nio.file.Paths;
 import java.util.Locale;
-import org.apache.lucene.analysis.ja.dict.DictionaryConstants;
 
 /**
  * Tool to build dictionaries. Usage:

lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/DictionaryConstants.java

@@ -17,7 +17,7 @@
 package org.apache.lucene.analysis.ja.dict;
 
 /** Dictionary constants */
-public final class DictionaryConstants {
+final class DictionaryConstants {
   /** Codec header of the dictionary file. */
   public static final String DICT_HEADER = "kuromoji_dict";
   /** Codec header of the dictionary mapping file. */

lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/util/ToStringUtil.java → lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/ToStringUtil.java

@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-package org.apache.lucene.analysis.ja.util;
+package org.apache.lucene.analysis.ja.dict;
 
 import java.io.IOException;
 import java.util.HashMap;