From 8f177d7951364cdae52c4b062274e77ba228c2bf Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 27 Mar 2024 10:53:16 +0100 Subject: [PATCH 1/2] Recommend lowering the default mmap readahead. This is a follow-up of a discussion on #13219. `mmap` has a higher readahead than regular `read()` operations by default, e.g. 128kB instead of 16kB on my Linux box. On indexes that exceed the size of the page cache, this may trigger performance issues due to page cache trashing and additional page cache contention. Rather than forcing `MMapDirectory` to use `MADV_RANDOM` on all files, it would make more sense to configure a lower `mmap` readahead at the system level, e.g. the same readahead value as `read()` operations use. --- .../java/org/apache/lucene/store/MMapDirectory.java | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java b/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java index c7e3518ab356..4fefcf2ff94d 100644 --- a/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java +++ b/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java @@ -38,6 +38,16 @@ * fragmented address space. If you get an {@link IOException} about mapping failed, it is * recommended to reduce the chunk size, until it works. * + *

NOTE: On some platforms like Linux, mmap comes with a higher readahead than regular + * read() operations, e.g. 128kB for mmap reads and 16kB for regular reads. Such a high default + * readahead can hurt performance on indexes that exceed the size of the page cache, by triggering + * more page cache trashing as well as more contention on the page cache. For typical workloads + * involving Lucene indexes, it is recommended to update the default readahead of mmap to the same + * readahead that is used by regular read() operations. This can be done by calling + * sudo blockdev --setra 32 <filesystem>, where 32 is the number of 512 byte + * sectors to read ahead, ie. 16kB, and <filesystem> is the filesystem where Lucene indexes + * are stored. + * *

This class supports preloading files into physical memory upon opening. This can help improve * performance of searches on a cold page cache at the expense of slowing down opening an index. See * {@link #setPreload(BiPredicate)} for more details. From 3dde0d866f980e52bf2bf7bfa93b68e948811f68 Mon Sep 17 00:00:00 2001 From: Adrien Grand Date: Wed, 27 Mar 2024 14:58:35 +0100 Subject: [PATCH 2/2] tidy --- .../src/java/org/apache/lucene/store/MMapDirectory.java | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java b/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java index 4fefcf2ff94d..dfc03cb01db2 100644 --- a/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java +++ b/lucene/core/src/java/org/apache/lucene/store/MMapDirectory.java @@ -43,10 +43,9 @@ * readahead can hurt performance on indexes that exceed the size of the page cache, by triggering * more page cache trashing as well as more contention on the page cache. For typical workloads * involving Lucene indexes, it is recommended to update the default readahead of mmap to the same - * readahead that is used by regular read() operations. This can be done by calling - * sudo blockdev --setra 32 <filesystem>, where 32 is the number of 512 byte - * sectors to read ahead, ie. 16kB, and <filesystem> is the filesystem where Lucene indexes - * are stored. + * readahead that is used by regular read() operations. This can be done by calling + * sudo blockdev --setra 32 <filesystem>, where 32 is the number of 512 byte sectors to + * read ahead, ie. 16kB, and <filesystem> is the filesystem where Lucene indexes are stored. * *

This class supports preloading files into physical memory upon opening. This can help improve * performance of searches on a cold page cache at the expense of slowing down opening an index. See