DRAFT - do not review #145
Conversation
Signed-off-by: Marc Handalian <handalm@amazon.com>
| override fun minTranslogGenRequired(readers: List<TranslogReader>, writer: TranslogWriter): Long { | ||
| val minBySize: Long = getMinTranslogGenBySize(readers, writer, retentionSizeInBytes) | ||
| val minByRetentionLeases: Long = getMinTranslogGenByRetentionLease(readers, writer) | ||
| val minByTranslogGenSettings = super.minTranslogGenRequired(readers, writer) |
There was a problem hiding this comment.
In this path getMinTranslogGenBySize will be invoked twice, once here and once by super. minTranslogGenRequired if this is too expensive, can revert back to updating the entire method.
…tionPolicy. This change allows for a custom deletion policy when setting: INDEX_TRANSLOG_RETENTION_LEASE_PRUNING_ENABLED_SETTING is enabled that prunes the translog based on retention leases. Signed-off-by: Marc Handalian <handalm@amazon.com>
|
Engine is very high level abstraction and one index can have only one engine which makes it very restrictive. https://github.com/opensearch-project/k-NN/blob/main/src/main/java/org/opensearch/knn/plugin/KNNPlugin.java. I was thinking whether we can provide granular extension points from InternalEngine to the plugins so that we can extend specific things in plugins rather than replacing the engine itself. We want to avoid the requirement of setting the engine as replicationengine in the leader index since that index might already have other engine set. Another requirement is that we should be able to make the current indices eligible for replication without imposing any downtime on it. |
+1 - created OpenSearch#1254 to track this better. One idea in the above direction is to provide a capability to provide TranslogDeletionPolicy alone. Multiple plugins can provide this. While applying, all policies are invoked and pruned until the min of all. |
|
Since we are creating actual PRs, closing this. |
Description
[Describe what this change achieves]
Issues Resolved
[List any issues this PR will resolve]
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.