Skip to content

Commit

Permalink
Add Hibernate Search management endpoint
Browse files Browse the repository at this point in the history
- to reindex either all or subset of indexed entities
  • Loading branch information
marko-bekhta committed Sep 15, 2023
1 parent 72a655e commit 5a33f53
Show file tree
Hide file tree
Showing 20 changed files with 955 additions and 1 deletion.
106 changes: 106 additions & 0 deletions docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -1015,6 +1015,112 @@ You can enable AWS request signing in Hibernate Search by adding a dedicated ext
See link:{hibernate-search-orm-elasticsearch-aws-guide}#aws-configuration-reference[the documentation for the Hibernate Search ORM + Elasticsearch AWS extension]
for more information.

[[management]]
== Management endpoint

[CAUTION]
====
Hibernate Search's management endpoint is considered preview.
In _preview_, backward compatibility and presence in the ecosystem is not guaranteed.
Specific improvements might require changing configuration or APIs, or even storage formats,
and plans to become _stable_ are under way.
Feedback is welcome on our https://groups.google.com/d/forum/quarkus-dev[mailing list]
or as issues in our https://github.com/quarkusio/quarkus/issues[GitHub issue tracker].
====

The Hibernate Search extension provides an HTTP endpoint to reindex your data through the xref:./management-interface-reference.adoc[management interface].
By default, this endpoint is not available. It can be enabled through configuration properties as shown below.

[source,properties]
----
quarkus.management.enabled=true <1>
quarkus.hibernate-search-orm.management.enabled=true <2>
----
<1> Enable the xref:./management-interface-reference.adoc[management interface].
<2> Enable Hibernate Search specific management endpoints.

Once the management is enabled, data can be re-indexed via `/q/hibernate-search/reindex`, where `/q` is the default management root path
and `/hibernate-search` is the default Hibernate Search root management path.
It (`/hibernate-search`) can be changed via configuration property as shown below.

[source,properties]
----
quarkus.hibernate-search-orm.management.root-path=custom-root-path <1>
----
<1> Use a custom `custom-root-path` path for Hibernate Search's management endpoint.
If the default management root path is used then the reindex path becomes `/q/custom-root-path/reindex`.

This endpoint accepts `POST` requests with `application/json` content type only.
All indexed entities will be re-indexed if an empty request body is submitted.
If only a subset of entities must be re-indexed or
if there is a need to have a custom configuration of the underlying mass indexer
then this information can be passed through the request body as shown below.

[source,json]
----
{
"filter": {
"types": ["EntityName1", "EntityName2", "EntityName3", ...], <1>
},
"massIndexer":{
"typesToIndexInParallel": 1, <2>
}
}
----
<1> An array of entity names that should be re-indexed. If unspecified or empty, all entity types will be re-indexed.
<2> Sets the number of entity types to be indexed in parallel.

The full list of possible filters and available mass indexer configurations is presented in the example below.

[source,json]
----
{
"filter": { <1>
"types": ["EntityName1", "EntityName2", "EntityName3", ...], <2>
"tenants": ["tenant1", "tenant2", ...] <3>
},
"massIndexer":{ <4>
"typesToIndexInParallel": 1, <5>
"threadsToLoadObjects": 6, <6>
"batchSizeToLoadObjects": 10, <7>
"cacheMode": "IGNORE", <8>
"mergeSegmentsOnFinish": false, <9>
"mergeSegmentsAfterPurge": true, <10>
"dropAndCreateSchemaOnStart": false, <11>
"purgeAllOnStart": true, <12>
"idFetchSize": 100, <13>
"transactionTimeout": 100000, <14>
}
}
----
<1> Filter object that allows to limit the scope of reindexing.
<2> An array of entity names that should be re-indexed. If unspecified or empty, all entity types will be re-indexed.
<3> An array of tenant ids, in case of multi-tenancy. If unspecified or empty, all tenants will be re-indexed.
<4> Mass indexer configuration object.
<5> Sets the number of entity types to be indexed in parallel.
<6> Sets the number of threads to be used to load the root entities.
<7> Sets the batch size used to load the root entities.
<8> Sets the cache interaction mode for the data loading tasks.
<9> Whether each index is merged into a single segment after indexing.
<10> Whether each index is merged into a single segment after the initial index purge, just before indexing.
<11> Whether the indexes and their schema (if they exist) should be dropped and re-created before indexing.
<12> Whether all entities are removed from the indexes before indexing.
<13> Specifies the fetch size to be used when loading primary keys if objects to be indexed.
<14> Specifies the timeout of transactions for loading ids and entities to be re-indexed.
+
Note all the properties in the json are optional, and only those that are needed should be used.

For more detailed information on mass indexer configuration see the
link:{hibernate-search-docs-url}#indexing-massindexer-parameters[corresponding section of the Hibernate Search reference documentation].

Submitting the reindexing request will trigger indexing in the background. Mass indexing progress will appear in the application logs.
For testing purposes, it might be useful to know when the indexing finished. Adding `wait_for=finished` query parameter to the URL
will result in the management endpoint returning a chunked response that will report when the indexing starts and then when it is finished.

When working with multiple persistence units, the name of the persistence unit to reindex can be supplied through the
`persistence_unit` query parameter: `/q/hibernate-search/reindex?persistence_unit=non-default-persistence-unit`.

== Further reading

If you are interested in learning more about Hibernate Search 6,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-vertx-http-dev-ui-spi</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-vertx-http-deployment</artifactId>
<optional>true</optional>
</dependency>

<!-- test dependencies -->
<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,11 @@
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchBuildTimeConfigPersistenceUnit.ElasticsearchIndexBuildTimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchRecorder;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchRuntimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.management.HibernateSearchManagementConfig;
import io.quarkus.runtime.configuration.ConfigUtils;
import io.quarkus.runtime.configuration.ConfigurationException;
import io.quarkus.vertx.http.deployment.NonApplicationRootPathBuildItem;
import io.quarkus.vertx.http.deployment.RouteBuildItem;

@BuildSteps(onlyIf = HibernateSearchEnabled.class)
class HibernateSearchElasticsearchProcessor {
Expand Down Expand Up @@ -435,4 +438,22 @@ void devServicesDropAndCreateAndDropByDefault(
}
}

@Record(ExecutionTime.RUNTIME_INIT)
@BuildStep(onlyIf = HibernateSearchManagementEnabled.class)
void createManagementRoutes(BuildProducer<RouteBuildItem> routes,
NonApplicationRootPathBuildItem nonApplicationRootPathBuildItem,
HibernateSearchElasticsearchRecorder recorder,
HibernateSearchManagementConfig managementConfig) {

routes.produce(nonApplicationRootPathBuildItem.routeBuilder()
.management()
.route(
managementConfig.rootPath() + (managementConfig.rootPath().endsWith("/") ? "" : "/")
+ "reindex")
.routeConfigKey("quarkus.hibernate-search-orm.management.root-path")
.handler(recorder.managementHandler())
.displayOnNotFoundPage()
.blockingRoute()
.build());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package io.quarkus.hibernate.search.orm.elasticsearch.deployment;

import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchBuildTimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.management.HibernateSearchManagementConfig;

/**
* Supplier that can be used to only run build steps
* if the Hibernate Search extension and its management is enabled.
*/
public class HibernateSearchManagementEnabled extends HibernateSearchEnabled {

private final HibernateSearchManagementConfig config;

HibernateSearchManagementEnabled(HibernateSearchElasticsearchBuildTimeConfig config,
HibernateSearchManagementConfig managementConfig) {
super(config);
this.config = managementConfig;
}

@Override
public boolean getAsBoolean() {
return super.getAsBoolean() && config.enabled();
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,12 @@
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchRuntimeConfigPersistenceUnit.ElasticsearchBackendRuntimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchRuntimeConfigPersistenceUnit.ElasticsearchIndexRuntimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.bean.HibernateSearchBeanUtil;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.management.HibernateSearchManagementHandler;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.mapping.QuarkusHibernateOrmSearchMappingConfigurer;
import io.quarkus.runtime.annotations.Recorder;
import io.quarkus.runtime.configuration.ConfigurationException;
import io.vertx.core.Handler;
import io.vertx.ext.web.RoutingContext;

@Recorder
public class HibernateSearchElasticsearchRecorder {
Expand Down Expand Up @@ -165,6 +168,10 @@ public SearchSession get() {
};
}

public Handler<RoutingContext> managementHandler() {
return new HibernateSearchManagementHandler();
}

private static final class HibernateSearchIntegrationStaticInitInactiveListener
implements HibernateOrmIntegrationStaticInitListener {
private HibernateSearchIntegrationStaticInitInactiveListener() {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
package io.quarkus.hibernate.search.orm.elasticsearch.runtime.management;

import io.quarkus.runtime.annotations.ConfigPhase;
import io.quarkus.runtime.annotations.ConfigRoot;
import io.smallrye.config.ConfigMapping;
import io.smallrye.config.WithDefault;

@ConfigMapping(prefix = "quarkus.hibernate-search-orm.management")
@ConfigRoot(phase = ConfigPhase.BUILD_AND_RUN_TIME_FIXED)
public interface HibernateSearchManagementConfig {

/**
* Root path for reindexing endpoints.
* This value will be resolved as a path relative to `${quarkus.management.root-path}`.
*
* @asciidoclet
*/
@WithDefault("hibernate-search/")
String rootPath();

/**
* If management interface is turned on the reindexing endpoints will be published under the management interface.
* This property allows to enable this functionality by setting it to ``true`.
*
* @asciidoclet
*/
@WithDefault("false")
boolean enabled();

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
package io.quarkus.hibernate.search.orm.elasticsearch.runtime.management;

import java.util.Locale;

import io.quarkus.arc.Arc;
import io.quarkus.arc.ManagedContext;
import io.vertx.core.Handler;
import io.vertx.core.http.HttpHeaders;
import io.vertx.core.http.HttpMethod;
import io.vertx.core.http.HttpServerRequest;
import io.vertx.ext.web.RoutingContext;

public class HibernateSearchManagementHandler implements Handler<RoutingContext> {

@Override
public void handle(RoutingContext routingContext) {
ManagedContext requestContext = Arc.container().requestContext();
if (requestContext.isActive()) {
doHandle(routingContext);
} else {
requestContext.activate();
try {
doHandle(routingContext);
} finally {
requestContext.terminate();
}
}
}

private void doHandle(RoutingContext ctx) {
HttpServerRequest request = ctx.request();

if (!HttpMethod.POST.equals(request.method())) {
errorResponse(ctx, 406, "Http method [" + request.method().name() + "] is not supported. Use [POST] instead.");
return;
}

String contentType = request.getHeader(HttpHeaders.CONTENT_TYPE);
if (contentType != null && !contentType.toLowerCase(Locale.ROOT).startsWith("application/json")) {
errorResponse(ctx, 406, "Content type [" + contentType + " is not supported. Use [application/json] instead.");
return;
}

new HibernateSearchPostRequestProcessor().process(ctx);
}

private void errorResponse(RoutingContext ctx, int code, String message) {
ctx.response()
.setStatusCode(code)
.setStatusMessage(message)
.end();
}
}
Loading

0 comments on commit 5a33f53

Please sign in to comment.