Limit concurrent S3 connections per host

This commit limits the number of concurrent connections to encourage
connection reuse. `MaxIdleConnsPerHost` was already set to workaround
golang/go#13801 ("issue which made Go consume
all ephemeral ports").

We observed a similar issue when starting up Loki (using weaveworks
`common/aws` throught Cortex) which resulted in thousands of very small
S3 requests until memcached was seeded. This failed because of ephemeral
port exhaustion: while the application in fact would have reused
connections, it already failed as it sent all of them concurrently until
most of the requests failed.

`MaxConnsPerHost` can be used to limit the overall number of parallel
connections.

aws/config.go

@@ -33,6 +33,7 @@ func ConfigFromURL(awsURL *url.URL) (*aws.Config, error) {
 				}).DialContext,
 				MaxIdleConns:          100,
 				IdleConnTimeout:       90 * time.Second,
+				MaxConnsPerHost:       100,
 				MaxIdleConnsPerHost:   100,
 				TLSHandshakeTimeout:   3 * time.Second,
 				ExpectContinueTimeout: 1 * time.Second,