Set LIBFFI_TMPDIR at startup #80651

DaveCTurner · 2021-11-11T08:56:56Z

Today if libffi cannot allocate pages of memory which are both
writeable and executable then it will attempt to write code to a
temporary file. Elasticsearch configures itself a suitable temporary
directory for use by JNA but by default libffi won't find this
directory and will try various other places. In certain configurations,
none of the other places that libffi tries are suitable. With older
versions of JNA this would result in a SIGSEGV; since #80617 the JVM
will exit with an exception.

With this commit we use the LIBFFI_TMPDIR environment variable to
configure libffi to use the same directory as JNA for its temporary
files if they are needed.

Closes #18272
Closes #73309
Closes #74545
Closes #77014
Closes #77053
Relates #77285

Co-authored-by: Rory Hunter roryhunter2@gmail.com

elasticmachine · 2021-11-11T16:38:01Z

Pinging @elastic/es-core-infra (Team:Core/Infra)

Today if `libffi` cannot allocate pages of memory which are both writeable and executable then it will attempt to write code to a temporary file. Elasticsearch configures itself a suitable temporary directory for use by JNA but by default `libffi` won't find this directory and will try various other places. In certain configurations, none of the other places that `libffi` tries are suitable. With older versions of JNA this would result in a `SIGSEGV`; since elastic#80617 the JVM will exit with an exception. With this commit we use the `LIBFFI_TMPDIR` environment variable to configure `libffi` to use the same directory as JNA for its temporary files if they are needed. Closes elastic#18272 Closes elastic#73309 Closes elastic#74545 Closes elastic#77014 Closes elastic#77053 Relates elastic#77285 Co-authored-by: Rory Hunter <roryhunter2@gmail.com>

pugnascotia · 2021-11-11T20:23:24Z

server/src/main/java/org/elasticsearch/env/Environment.java

        }
+        return PathUtils.get(workingDirectory).resolve(environmentVariable);


Is this line so that the env var can be either relative or absolute?

Really so that we report the absolute path that we expected to exist (and that libffi would use) even if the env var is a relative path. Elasticsearch's actual working directory is different from the working directory in which the user executes $ES_DIR/bin/elasticsearch so the behaviour can be surprising otherwise.

Today if `libffi` cannot allocate pages of memory which are both writeable and executable then it will attempt to write code to a temporary file. Elasticsearch configures itself a suitable temporary directory for use by JNA but by default `libffi` won't find this directory and will try various other places. In certain configurations, none of the other places that `libffi` tries are suitable. With older versions of JNA this would result in a `SIGSEGV`; since elastic#80617 the JVM will exit with an exception. With this commit we use the `LIBFFI_TMPDIR` environment variable to configure `libffi` to use the same directory as JNA for its temporary files if they are needed. Closes elastic#18272 Closes elastic#73309 Closes elastic#74545 Closes elastic#77014 Closes elastic#77053 Relates elastic#77285 Co-authored-by: Rory Hunter <roryhunter2@gmail.com>

Today if `libffi` cannot allocate pages of memory which are both writeable and executable then it will attempt to write code to a temporary file. Elasticsearch configures itself a suitable temporary directory for use by JNA but by default `libffi` won't find this directory and will try various other places. In certain configurations, none of the other places that `libffi` tries are suitable. With older versions of JNA this would result in a `SIGSEGV`; since #80617 the JVM will exit with an exception. With this commit we use the `LIBFFI_TMPDIR` environment variable to configure `libffi` to use the same directory as JNA for its temporary files if they are needed. Closes #18272 Closes #73309 Closes #74545 Closes #77014 Closes #77053 Relates #77285 Co-authored-by: Rory Hunter <roryhunter2@gmail.com> Co-authored-by: Rory Hunter <roryhunter2@gmail.com>

* Set LIBFFI_TMPDIR at startup (#80651) Today if `libffi` cannot allocate pages of memory which are both writeable and executable then it will attempt to write code to a temporary file. Elasticsearch configures itself a suitable temporary directory for use by JNA but by default `libffi` won't find this directory and will try various other places. In certain configurations, none of the other places that `libffi` tries are suitable. With older versions of JNA this would result in a `SIGSEGV`; since #80617 the JVM will exit with an exception. With this commit we use the `LIBFFI_TMPDIR` environment variable to configure `libffi` to use the same directory as JNA for its temporary files if they are needed. Closes #18272 Closes #73309 Closes #74545 Closes #77014 Closes #77053 Relates #77285 Co-authored-by: Rory Hunter <roryhunter2@gmail.com> * Fix incorrect SSL usage Co-authored-by: Rory Hunter <roryhunter2@gmail.com>

Today we suggest that users set `ES_TMPDIR` using `export`, which only works if you're running things directly from the shell. Yet most users encountering `ES_TMPDIR` problems seem to on RHEL and trying to run things via `systemd`, for whom the `export` suggestion doesn't work. This commit adds to the docs a suggestion of how to adjust the `systemd` service file to set the appropriate environment variable. Relates elastic#80651

Today we suggest that users set `ES_TMPDIR` using `export`, which only works if you're running things directly from the shell. Yet most users encountering `ES_TMPDIR` problems seem to on RHEL and trying to run things via `systemd`, for whom the `export` suggestion doesn't work. This commit adds to the docs a suggestion of how to adjust the `systemd` service file to set the appropriate environment variable. Relates #80651

Today we suggest that users set `ES_TMPDIR` using `export`, which only works if you're running things directly from the shell. Yet most users encountering `ES_TMPDIR` problems seem to on RHEL and trying to run things via `systemd`, for whom the `export` suggestion doesn't work. This commit adds to the docs a suggestion of how to adjust the `systemd` service file to set the appropriate environment variable. Relates elastic#80651

Today we suggest that users set `ES_TMPDIR` using `export`, which only works if you're running things directly from the shell. Yet most users encountering `ES_TMPDIR` problems seem to on RHEL and trying to run things via `systemd`, for whom the `export` suggestion doesn't work. This commit adds to the docs a suggestion of how to adjust the `systemd` service file to set the appropriate environment variable. Relates #80651

DaveCTurner added >bug WIP :Core/Infra/Core Core issues without another label labels Nov 11, 2021

elasticsearchmachine added the v8.1.0 label Nov 11, 2021

DaveCTurner force-pushed the 2021-11-11-set-LIBFFI_TMPDIR branch 2 times, most recently from e451699 to 9bad3ac Compare November 11, 2021 15:07

DaveCTurner added v7.16.1 v8.0.0 labels Nov 11, 2021

DaveCTurner force-pushed the 2021-11-11-set-LIBFFI_TMPDIR branch 2 times, most recently from 0b3cb94 to 24026d3 Compare November 11, 2021 16:37

DaveCTurner marked this pull request as ready for review November 11, 2021 16:37

elasticmachine added the Team:Core/Infra Meta label for core/infra team label Nov 11, 2021

DaveCTurner removed the WIP label Nov 11, 2021

DaveCTurner force-pushed the 2021-11-11-set-LIBFFI_TMPDIR branch from 24026d3 to b287852 Compare November 11, 2021 16:49

DaveCTurner force-pushed the 2021-11-11-set-LIBFFI_TMPDIR branch from b287852 to 92bd545 Compare November 11, 2021 16:53

DaveCTurner requested a review from pugnascotia November 11, 2021 19:06

pugnascotia reviewed Nov 11, 2021

View reviewed changes

pugnascotia approved these changes Nov 11, 2021

View reviewed changes

DaveCTurner added 3 commits November 12, 2021 12:45

Merge branch 'master' into 2021-11-11-set-LIBFFI_TMPDIR

004a007

Comment on why we resolve relative paths during validation

ee678a3

CI trigger

11e9614

DaveCTurner merged commit 57dbefe into elastic:master Nov 15, 2021

DaveCTurner deleted the 2021-11-11-set-LIBFFI_TMPDIR branch November 15, 2021 08:19

DaveCTurner mentioned this pull request Nov 15, 2021

[DOCS] RHEL Security TMP override #77285

Closed

danhermann added v7.16.0 and removed v7.16.1 labels Nov 16, 2021

mark-vieira added v8.0.0-rc1 and removed v8.0.0 labels Jan 12, 2022

DaveCTurner mentioned this pull request Jan 20, 2023

Document how to set ES_TMPDIR in the service file #93121

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set LIBFFI_TMPDIR at startup #80651

Set LIBFFI_TMPDIR at startup #80651

DaveCTurner commented Nov 11, 2021 •

edited

Loading

elasticmachine commented Nov 11, 2021

pugnascotia Nov 11, 2021

DaveCTurner Nov 12, 2021

		}
		return PathUtils.get(workingDirectory).resolve(environmentVariable);

Set LIBFFI_TMPDIR at startup #80651

Set LIBFFI_TMPDIR at startup #80651

Conversation

DaveCTurner commented Nov 11, 2021 • edited Loading

elasticmachine commented Nov 11, 2021

pugnascotia Nov 11, 2021

Choose a reason for hiding this comment

DaveCTurner Nov 12, 2021

Choose a reason for hiding this comment

DaveCTurner commented Nov 11, 2021 •

edited

Loading