Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Zgrab2 + TLS CLient Cert Auth causes failure, but curl and openssl s_client succeed #252

Closed
mzpqnxow opened this issue Feb 25, 2020 · 3 comments
Closed

Zgrab2 + TLS CLient Cert Auth causes failure, but curl and openssl s_client succeed #252

mzpqnxow opened this issue Feb 25, 2020 · 3 comments

Comments

@mzpqnxow
Copy link
Contributor

mzpqnxow commented Feb 25, 2020 •
edited
Loading

Zgrab2 + TLS CLient Cert Auth Causes Failure, But curl and openssl s_client Succeed

This bit about golang and session renegotiation shines some light on things
This might be helpful enough for me to produce a PR

So I found a bit of a strange edge-case with how zgrab2 handles TLS client certificate authentication failure in certain server/load-balancer configurations. In short, curl, openssl s_client -connect, etc. retrieve HTML content but zgrab2 does not; it returns an empty response (see below)

The example site is https://cn-stg-mbr-migration-websvc-scus.azurewebsites.net
The error emitted from zgrab2 is local error: no renegotiation. More detail is below

Curl Behavior - Successfully retrieves HTML

Perform a standard HTTP 1.1 GET request on the server. It successfully retrieves HTML, though it contains an error message.

$ curl --http1.1 -k https://cn-stg-mbr-migration-websvc-scus.azurewebsites.net 2>/dev/null
<!DOCTYPE html>
<html>
<head>
    <title>Web App - Unavailable</title>
...
<body bgcolor="#00abec">
    <div id="feature">
        <div id="content">
            <h1 id="unavailable">Error 403 - Forbidden: Client Certificate Required.</h1>
            <p id="tryAgain">The web app you have attempted to reach requires a client certificate and does not accept any requests without one.</p>
        </div>
    </div>
</body>
</html>
$

Zgrab2 Behavior - Fails to retreive HTML

When using zgrab2 to try to retrieve this HTML content, it fails. No response data is set and the error is described as local error: no renegotiation

› echo cn-stg-mbr-migration-websvc-scus.azurewebsites.net | zgrab2 http -p 443 --use-https --max-redirects=5 2>/dev/null
{"domain":"cn-stg-mbr-migration-websvc-scus.azurewebsites.net","data":{"http":{"status":"unknown-error","protocol":"http","result":{},"timestamp":"2020-02-25T13:19:51-05:00","error":"local error: no renegotiation"}}}

I fiddled with this for a bit, and tried explicitly disabling SNI (--no-sni) and with explicitly setting the SNI and/or Host header using --server-name=. No luck. I even tried --follow-localhost-redirects, just to make sure there wasn't some confused logic I could work around.

Perhaps there is some parameter missing when using the golang TLS libraries, or maybe there's logic in zgrab2 code causing it to drop the HTTP response. I haven't looked closely at the code yet but will contribute any more detail here that I can if I get some time to do so.

SUMMARY: I believe the desirable behavior for zgrab2 should be identical to how curl handles it, producing the above HTML as a response, rather than aborting and returning {} as a result
@mzpqnxow
Copy link
Contributor Author

Looking a little more deeply into this led to this code in zcrypto

		// Not merged from upstream:
		// GetCertificate: c.GetCertificate,
		// DynamicRecordSizingDisabled: c.DynamicRecordSizingDisabled,
		// VerifyPeerCertificate:    c.VerifyPeerCertificate,
		// KeyLogWriter:             c.KeyLogWriter,
		// Renegotiation:            c.Renegotiation,

It seems that the correct way to enable renegotiation would be to use:
RenegotiateFreelyAsClient or

See also here which shows in the golang TLS code the options available for renegotiation:

	RenegotiateNever RenegotiationSupport = iota
	// RenegotiateOnceAsClient allows a remote server to request
	// renegotiation once per connection.
	RenegotiateOnceAsClient
	// RenegotiateFreelyAsClient allows a remote server to repeatedly
	// request renegotiation.
	RenegotiateFreelyAsClient

It seems that (probably for security reasons?) zmap/zcrypto/zgrab2 made a choice to not allow renegotiation. It seems to me that it would be more appropriate to allow the user to turn it on if they want it

@mzpqnxow mzpqnxow mentioned this issue Mar 2, 2020
Closed
@mzpqnxow
Copy link
Contributor Author

mzpqnxow commented Mar 3, 2020

Note, this has been take up as a legitimate TODO for zcrypto @ zmap/zcrypto#212

Once that is handled, I am happy to PR this functionality to zgrab2 as an optional (off by default) command-line flag

@mzpqnxow mzpqnxow mentioned this issue Jul 10, 2020
Closed
@mzpqnxow mzpqnxow mentioned this issue Dec 6, 2020
Closed
@mzpqnxow
Copy link
Contributor Author

Thanks to the work @dissoupov did on the TLS1.3 branch of zcrypto, I have a PR that implements a fix for this via a flag, --renegotiate-freely

I'd appreciate feedback from any zgrab2 devs on this- the "default" configuration of golang TLS stack is to not allow server-requested renegotiations at all. There are 3 values for this in the golang tls config:

  1. Don't allow renegotiation requested by the server during the handshake at all
  2. Allow it to occur once
  3. Allow it to occur "freely" (any time the server pleases, I guess within whatever the RFC allows)

So my question is, should the default in zgrab2 be to allow it once? Allow it freely? Or should it remain as I have implemented it, where the user must explicitly ask for it, via --renegotiate-freely?

I think options 2 or 3 would be best- those in infosec for a long time are paranoid about allowing (arguably) unnecessary and complexity introducing behaviors like this into code, but because zgrab2 is more for data gathering than it is for secure browsing, I don't think there should be a concern about a future crypto attack associated with allowing one (or unlimited) renegotiation requests in the handshake

The branch is here: https://github.com/mzpqnxow/zgrab2/tree/feature/renegotiate

@mzpqnxow mzpqnxow mentioned this issue Nov 16, 2021
Closed
@mzpqnxow mzpqnxow mentioned this issue Apr 13, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mzpqnxow