Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RHEL6: IcingaWeb2 (v2.7.1) via API to Icinga2 (v2.11) => SSL Error / no shared cipher #7501

Closed
distahl opened this issue Sep 16, 2019 · 10 comments · Fixed by #7503
Closed

RHEL6: IcingaWeb2 (v2.7.1) via API to Icinga2 (v2.11) => SSL Error / no shared cipher #7501

distahl opened this issue Sep 16, 2019 · 10 comments · Fixed by #7503
Assignees
Labels
area/api REST API bug Something isn't working
Milestone

Comments

@distahl
Copy link

distahl commented Sep 16, 2019

Describe the bug

Since updating Icinga2 to v2.11.0.rc1.143.g358d951-0.20190913.1724, IcingaWeb2 isn't able to connet to it using the API.

-- icinga2.log --
[2019-09-16 08:19:29 +0200] critical/ApiListener: Client TLS handshake failed (from [1.2.3.4]:50470): no shared cipher

-- curl --

* About to connect() to 1.2.3.4 port 5665 (#0)
*   Trying 1.2.3.4... connected
* Connected to 1.2.3.4 (1.2.3.4) port 5665 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* NSS error -5938
* Closing connection #0
* SSL connect error
curl: (35) SSL connect error

To Reproduce

Just configure Monitoring Module to use API or use curl, e.g. like this:
curl -v -k -u user:pass 'https://1.2.3.4:5665/v1/status/IcingaApplication?pretty=1'

Expected behavior

Connect to API without issues

Your Environment

  • IcingaWeb2 v2.7.1 on RHEL 6.10 (Server A)
  • Icinga2 v2.11.0.rc1.143.g358d951-0.20190913.1724 on RHEL 6.10 (Server B)
  • Some, maybe, interesting packages installed on IcingaWeb2 Server:
nss.x86_64                                                                    3.36.0-9.el6_10
curl.x86_64                                                                   7.19.7-53.el6_9
libcurl.x86_64                                                                7.19.7-53.el6_9
openssl.x86_64                                                                1.0.1e-57.el6
rh-php70.x86_64                                                               2.3-1.el6

Additional context

  • As this bug also applies to Icinga2, I wasn't sure if I should post it here or in Icinga2. So feel free to move it wherever it suits you :-)
  • I know that Icinga2 changed the minimum TSL Version and Ciphers allowed by the API. So this is most likely the the cause of this.
  • Of course I could change the allowed ciphers in the Icinga2 Config, but as you officially support RHEL6, shouldn't you make the config compatible with older OS-Versions/Packages by default?
@dnsmichi
Copy link
Contributor

Hi,

can you run sslscan against the API and extract which cipher suite is required to properly connect on el6? For el7 clients, DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 were added already.

Cheers,
Michael

@dnsmichi dnsmichi transferred this issue from Icinga/icingaweb2 Sep 16, 2019
@dnsmichi dnsmichi added needs feedback We'll only proceed once we hear from you again area/api REST API labels Sep 16, 2019
@distahl
Copy link
Author

distahl commented Sep 16, 2019

Hi,

This is the result of the scan:

sslscan --no-failed --tls12 1.2.3.4:5665
                   _
           ___ ___| |___  ___ __ _ _ __
          / __/ __| / __|/ __/ _` | '_ \
          \__ \__ \ \__ \ (_| (_| | | | |
          |___/___/_|___/\___\__,_|_| |_|

                sslscan version 1.10.2 
                OpenSSL 1.0.1e-fips 11 Feb 2013


Testing SSL server 1.2.3.4 on port 5665

  Supported Client Cipher(s):
    ECDHE-RSA-AES256-GCM-SHA384
    ECDHE-ECDSA-AES256-GCM-SHA384
    ECDHE-RSA-AES256-SHA384
    ECDHE-ECDSA-AES256-SHA384
    ECDHE-RSA-AES256-SHA
    ECDHE-ECDSA-AES256-SHA
    DHE-DSS-AES256-GCM-SHA384
    DHE-RSA-AES256-GCM-SHA384
    DHE-RSA-AES256-SHA256
    DHE-DSS-AES256-SHA256
    DHE-RSA-AES256-SHA
    DHE-DSS-AES256-SHA
    DHE-RSA-CAMELLIA256-SHA
    DHE-DSS-CAMELLIA256-SHA
    AECDH-AES256-SHA
    ADH-AES256-GCM-SHA384
    ADH-AES256-SHA256
    ADH-AES256-SHA
    ADH-CAMELLIA256-SHA
    ECDH-RSA-AES256-GCM-SHA384
    ECDH-ECDSA-AES256-GCM-SHA384
    ECDH-RSA-AES256-SHA384
    ECDH-ECDSA-AES256-SHA384
    ECDH-RSA-AES256-SHA
    ECDH-ECDSA-AES256-SHA
    AES256-GCM-SHA384
    AES256-SHA256
    AES256-SHA
    CAMELLIA256-SHA
    PSK-AES256-CBC-SHA
    ECDHE-RSA-AES128-GCM-SHA256
    ECDHE-ECDSA-AES128-GCM-SHA256
    ECDHE-RSA-AES128-SHA256
    ECDHE-ECDSA-AES128-SHA256
    ECDHE-RSA-AES128-SHA
    ECDHE-ECDSA-AES128-SHA
    DHE-DSS-AES128-GCM-SHA256
    DHE-RSA-AES128-GCM-SHA256
    DHE-RSA-AES128-SHA256
    DHE-DSS-AES128-SHA256
    DHE-RSA-AES128-SHA
    DHE-DSS-AES128-SHA
    DHE-RSA-SEED-SHA
    DHE-DSS-SEED-SHA
    DHE-RSA-CAMELLIA128-SHA
    DHE-DSS-CAMELLIA128-SHA
    AECDH-AES128-SHA
    ADH-AES128-GCM-SHA256
    ADH-AES128-SHA256
    ADH-AES128-SHA
    ADH-SEED-SHA
    ADH-CAMELLIA128-SHA
    ECDH-RSA-AES128-GCM-SHA256
    ECDH-ECDSA-AES128-GCM-SHA256
    ECDH-RSA-AES128-SHA256
    ECDH-ECDSA-AES128-SHA256
    ECDH-RSA-AES128-SHA
    ECDH-ECDSA-AES128-SHA
    AES128-GCM-SHA256
    AES128-SHA256
    AES128-SHA
    SEED-SHA
    CAMELLIA128-SHA
    PSK-AES128-CBC-SHA
    ECDHE-RSA-DES-CBC3-SHA
    ECDHE-ECDSA-DES-CBC3-SHA
    EDH-RSA-DES-CBC3-SHA
    EDH-DSS-DES-CBC3-SHA
    AECDH-DES-CBC3-SHA
    ADH-DES-CBC3-SHA
    ECDH-RSA-DES-CBC3-SHA
    ECDH-ECDSA-DES-CBC3-SHA
    DES-CBC3-SHA
    IDEA-CBC-SHA
    PSK-3DES-EDE-CBC-SHA
    KRB5-IDEA-CBC-SHA
    KRB5-DES-CBC3-SHA
    KRB5-IDEA-CBC-MD5
    KRB5-DES-CBC3-MD5
    ECDHE-RSA-RC4-SHA
    ECDHE-ECDSA-RC4-SHA
    AECDH-RC4-SHA
    ADH-RC4-MD5
    ECDH-RSA-RC4-SHA
    ECDH-ECDSA-RC4-SHA
    RC4-SHA
    RC4-MD5
    PSK-RC4-SHA
    KRB5-RC4-SHA
    KRB5-RC4-MD5
    ECDHE-RSA-NULL-SHA
    ECDHE-ECDSA-NULL-SHA
    AECDH-NULL-SHA
    ECDH-RSA-NULL-SHA
    ECDH-ECDSA-NULL-SHA
    NULL-SHA256
    NULL-SHA
    NULL-MD5
  Supported Server Cipher(s):
    Accepted  TLS12  256 bits  AES256-GCM-SHA384

  Preferred Server Cipher(s):
    TLS12  256 bits  AES256-GCM-SHA384

Hmm, so if I get this right. Sslscan is telling me we are having a working cipher (AES256-GCM-SHA384), but still, it isn't working.

@distahl
Copy link
Author

distahl commented Sep 16, 2019

phpinfo() tells me the following for the curl module:

curl

cURL support => enabled
cURL Information => 7.19.7
Age => 3
Features
AsynchDNS => No
CharConv => No
Debug => No
GSS-Negotiate => Yes
IDN => Yes
IPv6 => Yes
krb4 => No
Largefile => Yes
libz => Yes
NTLM => Yes
SPNEGO => No
SSL => Yes
SSPI => No
Protocols => tftp, ftp, telnet, dict, ldap, ldaps, http, file, https, ftps, scp, sftp
Host => x86_64-redhat-linux-gnu
SSL Version => NSS/3.27.1
ZLib Version => 1.2.3
libSSH Version => libssh2/1.4.2

So I guess we searching the wrong library. Looks like curl is using NSS and not Openssl.

@dnsmichi
Copy link
Contributor

In terms of support, the master server components should be running on the latest el7 distribution, to avoid possible shortcomings with years old software like this problem.

https://icinga.com/subscription/support-details/

In terms of the problem itself - the API itself works fine, and we're back with curl/NSS bugs.

Tests in Docker

docker run -ti centos:6 bash

yum -y install https://packages.icinga.com/epel/icinga-rpm-release-6-latest.noarch.rpm
yum -y install epel-release
yum makecache


yum install --enablerepo=icinga-snapshot-builds icinga2

yum -y install sslscan

icinga2 api setup

/usr/lib/icinga2/prepare-dirs /etc/sysconfig/icinga2
icinga2 daemon > /tmp/icinga2.log 2>&1 &


sslscan

sslscan localhost:5665


  Preferred Server Cipher(s):
    SSLv2  0 bits    (NONE)
    SSLv3  0 bits    (NONE)
    TLSv1  0 bits    (NONE)
    TLS11  0 bits    (NONE)
    TLS12  256 bits  AES256-GCM-SHA384

openssl

openssl s_client -connect localhost:5665

New, TLSv1/SSLv3, Cipher is AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-GCM-SHA384
    Session-ID: 8E45899661A36B3EBED4F7A7B938C5735F8DE976FF6F376B4212FB640021BDC9
    Session-ID-ctx:
    Master-Key: 8A02B2FEADCB8A6E69B2E7C7770F612D74D47724BF37E29CB6DBA4DF7CE98B9635820259A69BD5EE57ED324A0BFC0550
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:

curl

[root@ef3e0bf7f9ff /]# curl -k -vvvv -u root:icinga 'https://localhost:5665'
* About to connect() to localhost port 5665 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 5665 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* NSS error -5938
* Closing connection #0
* SSL connect error
curl: (35) SSL connect error

https://stackoverflow.com/questions/21887315/curl-ssl-connect-error-35-with-nss-error-5961/22135607

yum update -y nss curl libcurl

https://bugzilla.redhat.com/show_bug.cgi?id=1185708

vim /etc/icinga2/features-enabled/api.conf

  cipher_list = "ECDHE-RSA-AES128-GCM-SHA256"

kill -HUP $(pidof icinga2)

PHP and cURL on RHEL 6

yum -y install centos-release-scl

https://bugzilla.redhat.com/show_bug.cgi?id=1289205
https://bugzilla.redhat.com/show_bug.cgi?id=1272504

Conclusion

To me, it seems that curl, NSS and PHP on RHEL6 don't support TLSv1.2 natively.

This brings me back to an old thread I've found while raising the TLS limits.

https://bugzilla.redhat.com/show_bug.cgi?id=1153814

Kamil Dudka 2014-10-17 10:56:32 UTC
(In reply to Adam Williamson (Red Hat) from comment #0)
> curl: (35) Cannot communicate securely with peer: no common encryption
> algorithm(s).

TLS 1.2 is not enabled by default in curl/nss, see bug #994599 for details.

> Using -v, I can see curl's using its NSS backend (though ldd shows for some
> reason it's built against both nss and openssl).

NSS is used mainly to implement TLS in libcurl.  openssl gets leaded via libssh2, which uses openssl crypto to implement SCP/SFTP protocols.

I can backport the options of (lib)curl to enable TLS 1.2, but I am afraid that it will not help to resolve the issue with yum.  Changing libcurl's default in an already released Fedora seems risky, given the fact that we are about to drop the fallback to SSLv3 at the same time:

http://thread.gmane.org/gmane.comp.web.curl.library/43887

=> https://bugzilla.redhat.com/show_bug.cgi?id=994599

and this lovely thing.

https://serverfault.com/questions/703436/ssl-handshake-with-centos-curl-and-ecdhe

I have tried some other cipher suites, but none of them worked.

@distahl
Copy link
Author

distahl commented Sep 16, 2019

In terms of support, the master server components should be running on the latest el7 distribution, to avoid possible shortcomings with years old software like this problem.

https://icinga.com/subscription/support-details/

Oh! Thanks for this Support-Matrix. Didn't know that :-)

To me, it seems that curl, NSS and PHP on RHEL6 don't support TLSv1.2 natively.

Don't think it's a TLSv1.2 issue. I tried with "AES256-SHA256" (just a lucky shot) and it was working.

Best would be to find a way making a sslscan with nss. I have to search around a bit.

@distahl
Copy link
Author

distahl commented Sep 16, 2019

So those are the ones I was able to identify working with curl and TLSv1.2.
Tested polling the API of Icinga with cipher_list = "ALL"

Icinga2=EL6 & Curl=EL6
Icinga2=EL6 & Curl=EL7

OPENSSL_NAME                             CURL_NAME                                IANA_NAME                                          STATUS
DES-CBC3-SHA                             rsa_3des_sha                             TLS_RSA_WITH_3DES_EDE_CBC_SHA                      OK
RC4-MD5                                  rsa_rc4_128_md5                          TLS_RSA_WITH_RC4_128_MD5                           OK
RC4-SHA                                  rsa_rc4_128_sha                          TLS_RSA_WITH_RC4_128_SHA                           OK
AES128-SHA                               rsa_aes_128_sha                          TLS_RSA_WITH_AES_128_CBC_SHA                       OK
AES256-SHA                               rsa_aes_256_sha                          TLS_RSA_WITH_AES_256_CBC_SHA                       OK
AES128-GCM-SHA256                        rsa_aes_128_gcm_sha_256                  TLS_RSA_WITH_AES_128_GCM_SHA256                    OK

Icinga2=EL7 & Curl=EL6
Icinga2=EL7 & Curl=EL7

OPENSSL_NAME                             CURL_NAME                                IANA_NAME                                          STATUS
DES-CBC3-SHA                             rsa_3des_sha                             TLS_RSA_WITH_3DES_EDE_CBC_SHA                      OK
RC4-MD5                                  rsa_rc4_128_md5                          TLS_RSA_WITH_RC4_128_MD5                           OK
RC4-SHA                                  rsa_rc4_128_sha                          TLS_RSA_WITH_RC4_128_SHA                           OK
AES128-SHA                               rsa_aes_128_sha                          TLS_RSA_WITH_AES_128_CBC_SHA                       OK
AES256-SHA                               rsa_aes_256_sha                          TLS_RSA_WITH_AES_256_CBC_SHA                       OK
AES128-GCM-SHA256                        rsa_aes_128_gcm_sha_256                  TLS_RSA_WITH_AES_128_GCM_SHA256                    OK
ECDHE-RSA-RC4-SHA                        ecdhe_rsa_rc4_128_sha                    TLS_ECDHE_RSA_WITH_RC4_128_SHA                     OK
ECDHE-RSA-DES-CBC3-SHA                   ecdhe_rsa_3des_sha                       TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA                OK
ECDHE-RSA-AES128-SHA                     ecdhe_rsa_aes_128_sha                    TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA                 OK
ECDHE-RSA-AES256-SHA                     ecdhe_rsa_aes_256_sha                    TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA                 OK
ECDHE-RSA-AES128-GCM-SHA256              ecdhe_rsa_aes_128_gcm_sha_256            TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256              OK

@dnsmichi
Copy link
Contributor

What happens if you configure the api.conf file to use

cipher_list = "AES128-GCM-SHA256"

?

@dnsmichi
Copy link
Contributor

dnsmichi commented Sep 16, 2019

https://wiki.mozilla.org/Security/Server_Side_TLS#Old_backward_compatibility

ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384:AES128-GCM-SHA256

works for me. Question is if that also solves the problem with PHP curl and your web interface.

@distahl
Copy link
Author

distahl commented Sep 16, 2019

What happens if you configure the api.conf file to use

cipher_list = "AES128-GCM-SHA256"

?

https://wiki.mozilla.org/Security/Server_Side_TLS#Old_backward_compatibility

ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384:AES128-GCM-SHA256

works for me. Question is if that also solves the problem with PHP curl and your web interface.

Both ways are working just fine as long as AES128-GCM-SHA256 is in the list.
Master-Master communication is working.
IcingaWeb2 communication to Master is working (so php is also working).

@dnsmichi
Copy link
Contributor

Okidoki, that's what I've thought. Thanks for keeping up here, I'll modify this for 2.11 once more.

@dnsmichi dnsmichi added this to the 2.11.0 milestone Sep 16, 2019
@dnsmichi dnsmichi self-assigned this Sep 16, 2019
@dnsmichi dnsmichi added bug Something isn't working and removed needs feedback We'll only proceed once we hear from you again labels Sep 16, 2019
@dnsmichi dnsmichi mentioned this issue Sep 16, 2019
54 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api REST API bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants