MaxConcurrentConnections reached

[mcacker]

It would be useful if the dashboard would report if the max number of connections is reached. Currently it just shows as "loading...", whereas the http request is responding with "MaxConcurrentConnections reached: 5"

Mitchell

[benjchristensen]

Good idea.

[davidkarlsen]

Does this work properly? If I hit refresh a fee times in the UI I reach the limit and it stays there

[davidkarlsen]

Does this work properly? If I hit refresh a few times in the UI I reach the limit and it stays there

[mattrjacobs]

@davidkarlsen Here's how I tested:

1. Open up the dashboard in a browser and verify that it behaves normally
2. Use curl to open a connection to a direct metrics stream
3. Repeat 2) until I get a 503. This implies the metrics stream supports no more listeners.
4. Open the dashboard in a new tab and verify I see the new error messages.
5. Shut down all curls still open from 2).
6. Open dashboard in a new tab and verify that the metrics show up (and the error messages do not).

This sequence worked for me with master.

If you've got a specific set of steps for me to try, I'm happy to try to replicate.

[davidkarlsen]

Seems like I can refresh and it recovers on recent versions. Sorry for the noise. I'll be back if I can make it fail consistently.

[mattrjacobs]

No problem - I'm always happy to have confirmation that a new feature is being used and whether it works as I expected.

[mattrjacobs]

Closed due to inactivity

[draxly]

I still encounter "MaxConcurrentConnections reached: 5" even though I am using Hystrix 1.4.12. It works in the beginning but when I load the Hystrix Dashboard and refresh the browser window 4 times, it stops working wit the above mentioned message. The setup is using Spring Boot with the @EnableHystrix annotation.

[andrewe123]

I'm seeing similar issue as @draxly. com.netflix.hystrix:hystrix-metrics-event-stream:1.4.22 using Spring Boot @EnableCircuitBreaker

curl 5x curl http://localhost/app/hystrix.stream
stream is outputted
ctrl+C

on the 6th curl
{"timestamp":1460728335392,"status":503,"error":"Service Unavailable","message":"MaxConcurrentConnections reached: 5","path":"/app/hystrix.stream"}

It doesn't seem to be releasing the connections when they are closed

[mattrjacobs]

I just attempted to replicate this, but could not. Here's what I did (from tip-of-master):

• Run the hystrix-examples-webapp (../gradlew clean jettyRun from Hystrix/hystrix-examples-webapp)
• Load the demo page to get commands executing (Visit http://localhost:8989/hystrix-examples-webapp/ in a browser)
• In a single terminal window, get the metrics stream and close it repeatedly (I did it 10x). This was a curl -v -G "http://localhost:8989/hystrix-examples-webapp/hystrix.stream" followed by a Ctrl+C.

Since this works, it looks to me like connections are getting cleaned up properly. Can you see if there's anything materially different in your setup from the experiment above?

[andrewe123]

@mattrjacobs Thanks! I've managed to narrow the issue down to the web server now.

[matheus208]

@andrewe123 Have you found out what it is? I'm facing the same issue on our production servers (but not on our development servers)

[andrewe123]

@matheus208 it appeared to be a WebSphere bug, we switched to Tomcat and couldn't reproduce the issue.