HTTP/2 Security Vulnerabilities

[skylerto]

Security vulnerabilities were reported with HTTP/2 implementations, gRPC being one of them. I know that this software makes extensive use of gRPC for internal service communications and wanted to bring it up as it could effect some of our clients. Looking for feedback on how the automate team might be addressing these vulnerabilities.

https://www.kb.cert.org/vuls/id/605641/

Describe the bug

Exploiting gRPC connections can be exploited with denial-of-service attacks causing unexpected resource consumption and/or service failures.

To Reproduce

Implement any of the issues brought up in this document: https://www.kb.cert.org/vuls/id/605641/

Expected behavior

Services should mitigate DOS attacks.

Screenshots

N/A

Versions (please complete the following information):

N/A

Additional context

Following reports:

• https://github.com/Netflix/security-bulletins/blob/master/advisories/third-party/2019-002.md
• https://www.kb.cert.org/vuls/id/605641/

[srenatus]

Thanks for opening this issue! I believe we've addressed these with #1237. 🤔 Haven't we? 😃

[srenatus]

That last comment from me was re: the internal gRPC APIs. The other HTTP2 APIs, both internal and external, are being worked on:

• stdlib net/http [go] Bump version to 1.12.8 habitat-sh/core-plans#2971 (needs promotion ✔️)
  • rebuild components using that: Force rebuild of components using Go net/http #1252
• nginx: [nginx] bump: 1.17.2 -> 1.17.3 habitat-sh/core-plans#2970 (merged, needs promotion)
  • rebuild automate-load-balancer et al. -- see trigger rebuilds for core/nginx/1.17.3 #1254
• dex: [dex] bump v2.18.0 -> v2.19.0 habitat-sh/core-plans#2972 (merged, promoted, needs a rebuild when core/go is prompted, and the PR below ✔️)
  • automate-dex: bump core/dex (2.17.0 -> 2.18.0), adapt code #1243
• investigate notifications-service (elixir-based)
  • Update notifications service dependencies #2869

[srenatus]

☝️ sorry for that, we've crossed wires when I wanted to fix the typo 😄

[skylerto]

@srenatus thanks for the quick reply && typo fix!

Looks like 3 of the 8 reported CVEs were taken care of as part of the 1.23.0 grpc-go release. Those 3 being:

• CVE-2019-9512 (Ping Flood)
• CVE-2019-9514 (Reset Flood)
• CVE-2019-9515 (Settings Flood)

Another 3 are covered at the proxy/load balancer (nginx):

• CVE-2019-9511 (Data Dribble)
• CVE-2019-9513 (Resource Loop)
• CVE-2019-9516 (0-Length Headers Leak)

I'm not sure if the other CVEs not covered are incarnations of the 6 that were fixed these are still not discussed anywhere:

• CVE-2019-9517 (Internal Data Buffering)
• CVE-2019-9518 (Empty Frame Flooding)

Forgive me, I'm trying to sort through all this. 🙈

[srenatus]

Forgive me, I'm trying to sort through all this. 🙈

Oh, it's super helpful that you're doing, and sharing this! 🎈After all, you can imagine that we're frantically bumping dependencies in many places, and it's good to know someone is keeping calm, watchful eye on the situation 😉

[srenatus]

Update the latest harts published to the dev channel have been rebuilt, to

• include versions of core/nginx and core/dex that have the fixes
• use net/http from golang 1.12.8

So, this won't make it into Monday's release, I suspect; but it'll be in the next one.

[stevendanna]

I've added a TODO to look into notifications-service which is elixir based and thus wouldn't have a fix included in what we've pushed currently.

[stevendanna]

ninenines/cowboy#1398

[sdelano]

According to the cowboy issue linked above it appears that these issues have been addressed in that project. There's no commit linking to the fixes however, so we'll have to verify.

[susanev]

@skylerto we addressed the last part of this issue in the latest release
https://automate.chef.io/release-notes/?v=20200220011437

with this work
d438271

let us know if you have any more concerns

[skylerto]

Looks great, thanks so much @susanev!