Many contract function calls/transactions via Web3 will eventually cause TCP ephemeral port exhaustion

[hcaw]

Description:
Spamming Web3.js with a few thousand transactions or calls in quick succession will eventually (about 30 seconds) cause all available dynamic ports (16384 on Windows) to be consumed - however the vast majority are sitting in TIME_WAIT state (which can be confirmed by running netstat). The onreadystatechange event handler in the sendAsync function in HttpProvider will get called, but request.responseText is empty, causing an exception to be thrown, which is caught but results in an unhelpful error - error: Invalid JSON RPC response: ""

Steps to reproduce:
Execute many successive contract function calls via Web3

Actual Behaviour:
error: Invalid JSON RPC response: "" Error message is being passed into the web3 callback function.

Expected Behaviour:
If all dynamic ports are consumed, then either return a descriptive error message, or hold off until ports are freed by the operating system, or make sure ports are being properly closed after use.

Notes:
To anyone seeing the same issue, it can be temporarily resolved by setting a timeout to wait until the ports are released by the operating system, or by manually increasing available dynamic TCP ports.
For more info - https://blogs.technet.microsoft.com/askds/2008/10/29/port-exhaustion-and-you-or-why-the-netstat-tool-is-your-friend/

Versions:
Geth 1.6.5 - Clique PoA
Web3.js 0.18.4
Windows 10

[hcaw]

See similar issue here, albeit on a different project raiden-network/raiden#253

[frozeman]

The problem is that web3.js is a library running in node OR the browser, so it has no way to see system level port configurations. If you have any idea how we could improve the error messages, feel free to suggest.

[csokun]

HttpProvider makes use of xmlhttprequest which internally disable http.agent https://github.com/driverdan/node-XMLHttpRequest/blob/master/lib/XMLHttpRequest.js#L384

According to nodejs doc https://nodejs.org/docs/latest-v10.x/api/http.html#http_class_http_agent
By providing {agent: false} as an option to the http.get() or http.request() functions, a one-time use Agent with default options will be used for the client connection.

Injecting http.Agent to underline xmlhttprequest probably a way to go:

const httpAgent = new http.Agent({ keepAlive: true, maxSockets: 20  });

[nivida]

I've closed this issue because the last maintenance release of 0.20.x is v0.20.7.