Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HybridDirectory should open .dim files with mmap #48509

Closed
jpountz opened this issue Oct 25, 2019 · 3 comments
Closed

HybridDirectory should open .dim files with mmap #48509

jpountz opened this issue Oct 25, 2019 · 3 comments
Assignees
@jdconrad
Labels
:Distributed Indexing/Store Issues around managing unopened Lucene indices. If it touches Store.java, this is a likely label. >enhancement

Comments

@jpountz
Copy link
Contributor

jpountz commented Oct 25, 2019

This is a follow-up to https://issues.apache.org/jira/browse/LUCENE-8932. The BKD index is now off-heap when the index input is a ByteBufferIndexInput. So if we want the index to move off-heap with the default directory implementation when upgrading to Lucene 8.4, we should make HybridDirectory open dim files with mmap.

@jdconrad I'm assigning it to you if you don't mind since you worked on the Lucene change
@jpountz jpountz added >enhancement :Distributed Indexing/Store Issues around managing unopened Lucene indices. If it touches Store.java, this is a likely label. labels Oct 25, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Store)

@jdconrad
Copy link
Contributor

@jpountz Happy to work on this. Will likely have follow up question soon :)

@jdconrad jdconrad mentioned this issue Nov 18, 2019
Merged
jdconrad added a commit that referenced this issue Nov 19, 2019
@jdconrad
make dim files mmapped (#49272)
8e2a23a 
This change mmaps dim files in HybridDirectory to take advantage of off-
heap BKD trees. This is based off of (#48509) via 
(https://issues.apache.org/jira/browse/LUCENE-8932).
jdconrad added a commit that referenced this issue Nov 19, 2019
@jdconrad
make dim files mmapped (#49272)
14d2e79 
This change mmaps dim files in HybridDirectory to take advantage of off-
heap BKD trees. This is based off of (#48509) via 
(https://issues.apache.org/jira/browse/LUCENE-8932).
@jdconrad
Copy link
Contributor

Closed by #49272

This was referenced Feb 3, 2020
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jdconrad jdconrad

Labels
:Distributed Indexing/Store Issues around managing unopened Lucene indices. If it touches Store.java, this is a likely label. >enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jpountz @jdconrad @elasticmachine