Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: require --openssl-no-asm if no suitable assembler #20217

Closed
wants to merge 2 commits into from

Conversation

rvagg
Copy link
Member

@rvagg rvagg commented Apr 23, 2018

Also includes a refactor to extract error(), like warn() and make the messages a bit more consistent.

The main behaviour change here is to fail ./configure if you don't have an assembler new enough, which forces you to supply --openssl-no-asm rather than doing it implicitly. The cost of no-asm is that you get binaries that are potentially a lot slower, it's not something you should walk blindly in to. Discussion around this is primarily in #19944

I'd really like to get this in to Node 10.0.0 so we don't have to argue about the semver-major nature of this. I know it's very late but I'd been assuming this was already done since we talked about it. All of these versions are pretty low (e.g. gcc 4.9 comes with binutils 2.25 and we're asking for 2.23), so aside from Windows (where we have it opt-in via vcbuild.bat already) and some of the exotics, I don't believe many (or any?) would be experiencing this since they already need a fairly modern toolchain for the C++ features we're now needing to support.

When we go to OpenSSL 1.1.1 these minimums are going to rise, but, it's my understanding that OpenSSL will only use the maximum that it has so the fall-back isn't as painful and we'll just need to provide a warning that you have suboptimal (but not terrible) binaries. I don't fully understand the $avx conditionals in the OpenSSL code and need @shigeki @bnoordhuis or someone else to help me with this. If this isn't the case then we're in a bit of trouble because gas gets bumped to 2.26 (or maybe even 3.0) and we might need to consider doing it right now!

This PR fails on AIX, we're going to have to put --openssl-no-asm for Node 10+ there until we get a newer assembler.

@shigeki @bnoordhuis @danbev @jasnell @nodejs/build

@nodejs-github-bot nodejs-github-bot added the build Issues and PRs related to build files or the CI. label Apr 23, 2018
@shigeki
Copy link
Contributor

shigeki commented Apr 23, 2018

I prefer to provide asm supports for older assemblers rather than making configure failure. It can be semver-minor changes.

@rvagg
Copy link
Member Author

rvagg commented Apr 23, 2018

@shigeki if/when we have that we could remove this entirely, it's going to be much easier to remove than add back in something like this

@shigeki
Copy link
Contributor

shigeki commented Apr 23, 2018

@rvagg Okay. I think it is better to make this limited to x86 architectures because assembler version checks are made in x86_64 and x86.

ohtsu@obu:~/github/openssl/crypto$ git grep -i 'gnu assembler version'
aes/asm/aesni-mb-x86_64.pl:             =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
aes/asm/aesni-sha1-x86_64.pl:           =~ /GNU assembler version ([2-9]\.[0-9]+)/ &&
aes/asm/aesni-sha256-x86_64.pl:         =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
bn/asm/rsaz-avx2.pl:            =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
bn/asm/rsaz-x86_64.pl:          =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
bn/asm/x86_64-mont.pl:          =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
bn/asm/x86_64-mont5.pl:         =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
chacha/asm/chacha-x86.pl:                       =~ /GNU assembler version ([2-9]\.[0-9]+)/ &&
chacha/asm/chacha-x86_64.pl:            =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
ec/asm/ecp_nistz256-avx2.pl:            =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
ec/asm/ecp_nistz256-x86_64.pl:          =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
ec/asm/x25519-x86_64.pl:                =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
modes/asm/aesni-gcm-x86_64.pl:          =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
modes/asm/ghash-x86_64.pl:              =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
poly1305/asm/poly1305-x86.pl:                   =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
poly1305/asm/poly1305-x86_64.pl:                =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
sha/asm/sha1-586.pl:                    =~ /GNU assembler version ([2-9]\.[0-9]+)/ &&
sha/asm/sha1-mb-x86_64.pl:              =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
sha/asm/sha1-x86_64.pl:         =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
sha/asm/sha256-586.pl:                  =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
sha/asm/sha256-mb-x86_64.pl:            =~ /GNU assembler version ([2-9]\.[0-9]+)/) {
sha/asm/sha512-x86_64.pl:               =~ /GNU assembler version ([2-9]\.[0-9]+)/) {

This diff can be workable.

diff --git a/configure b/configure
index 0294e3a..c1170f9 100755
--- a/configure
+++ b/configure
@@ -1106,6 +1106,8 @@ def configure_openssl(o):
       o['defines'] += ['NODE_WITHOUT_NODE_OPTIONS']

   if not options.shared_openssl and not options.openssl_no_asm:
+    is_x86 = 'x64' in variables['target_arch'] or 'ia32' in variables['target_arch']
+
     # supported asm compiler for AVX2. See https://github.com/openssl/openssl/
     # blob/OpenSSL_1_1_0-stable/crypto/modes/asm/aesni-gcm-x86_64.pl#L52-L69
     openssl110_asm_supported = \
@@ -1114,7 +1116,7 @@ def configure_openssl(o):
       ('llvm_version' in variables and float(variables['llvm_version']) >= 3.3) or \
       ('nasm_version' in variables and float(variables['nasm_version']) >= 2.10)

-    if not openssl110_asm_supported:
+    if is_x86 and not openssl110_asm_supported:
       error('''Did not find a new enough assembler, install one or build with
        --openssl-no-asm.
        Please refer to BUILDING.md''')

@shigeki
Copy link
Contributor

shigeki commented Apr 23, 2018

Checking CI in https://ci.nodejs.org/job/node-test-commit/17948/ to see if AIX and others not x86 work fine with my additional patch.

@jasnell
Copy link
Member

jasnell commented Apr 23, 2018

This would need @nodejs/tsc approval to fast track. It would need to land before noon Pacific time today to make it in to 10.0.0.

@shigeki
Copy link
Contributor

shigeki commented Apr 23, 2018

Thre results of https://ci.nodejs.org/job/node-test-commit/17948/ are fine except debian8-64 where only one http2 flaky test was failed.
Both AIX and S390 can be built with asm support by using current aix assembler and their CI results are fine. This solves #19944 (comment).

# supported asm compiler for AVX2. See https://github.com/openssl/openssl/
# blob/OpenSSL_1_1_0-stable/crypto/modes/asm/aesni-gcm-x86_64.pl#L52-L69
openssl110_asm_supported = \
('gas_version' in variables and float(variables['gas_version']) >= 2.23) or \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is going to work on AIX as I believe we are not using gas or one of the other options. I think at this point we should just exclude the check on AIX as its so late in the game.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the suggestion from @shigeki, that makes sense to me.

@mhdawson
Copy link
Member

If we add the check I'm +1 for only doing the check for x86 platforms.

@shigeki
Copy link
Contributor

shigeki commented Apr 23, 2018

I submit a new PR of this with my additional commit in #20226.

jasnell pushed a commit that referenced this pull request Apr 23, 2018
PR-URL: #20226
Fixes: #19944
Refs: #20217
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
jasnell pushed a commit that referenced this pull request Apr 23, 2018
PR-URL: #20226
Fixes: #19944
Refs: #20217
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
jasnell pushed a commit that referenced this pull request Apr 23, 2018
The current openssl checks assembler version only x86_64 or ia32
target arch for use of AES-NI, AVX and AVX2.
This requires --openssl-no-asm option during configure when an older
assembler version is found only on x86_64 or ia32.

PR-URL: #20226
Fixes: #19944
Refs: #20217
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
jasnell pushed a commit that referenced this pull request Apr 23, 2018
PR-URL: #20226
Fixes: #19944
Refs: #20217
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
jasnell pushed a commit that referenced this pull request Apr 23, 2018
PR-URL: #20226
Fixes: #19944
Refs: #20217
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
jasnell pushed a commit that referenced this pull request Apr 23, 2018
The current openssl checks assembler version only x86_64 or ia32
target arch for use of AES-NI, AVX and AVX2.
This requires --openssl-no-asm option during configure when an older
assembler version is found only on x86_64 or ia32.

PR-URL: #20226
Fixes: #19944
Refs: #20217
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ali Ijaz Sheikh <ofrobots@google.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
@shigeki
Copy link
Contributor

shigeki commented Apr 23, 2018

Closed by #20226.

@shigeki shigeki closed this Apr 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants