How to set Keep-Alive to the Header to request to S3?

[skyleelove]

Hi,
I want to set a long connection to s3,how i can use the sdk to do this?Thanks a lot

[jasdel]

Hi @skyleelove, by default the SDK will use connection pooling and keep alive for all requests made to S3. These connections may be closed server side if they are not reused within a short duration though.

[skyleelove]

Hi,jasdel

Thanks for your answer.I get so many TIME_WAIT when I put object to S3 using the sdk for go.How can I do for solve the problem？

[jasdel]

Thanks for the update @skyleelove. Are you getting the TIME_WAIT for waits when putting just a single object to S3, or only with multiple file uploads? In addition, are you using the S3 Upload Manager or the S3.PutObject() api call? What version of Go and the SDK are you using?

If you have any sample code that reproduces the issue it may be helpful identifying the issue you're seeing.

[skyleelove]

Thanks a lot .
The version of go is go version go1.6 darwin/amd64
The version of sdk version v1.4.5
I use the S3.PutObject() and put 200 object to s3 one second

[skyleelove]

Now I update the sdk to v1.4.8 .It still has so mang TIME_WAIT

[jasdel]

Thanks for the update. How does your code initialize the S3 client, and is the s3 client instance shared between all of your upload requests? Or is a new client created for each upload?

The SDK's service clients are safe to shared among gorouties, so you shouldn't need to create a new client for each goroutine.

[skyleelove]

var s3Client *s3.S3
I just use one client for all goroutine

[skyleelove]

func init() {
config = &aws.Config{
Endpoint: aws.String(AWS_ENDPOINT),
Region: aws.String(AWS_REGION),
Credentials: credentials.NewStaticCredentials(AWS_ACCESS_KEY, AWS_SECRET_KEY, ""),
}
s3Client = s3.New(session.New(config))
}

Here are init client

[jasdel]

Do you see a TIME_WAIT for each sock for each put or only some of the puts? Its possible that S3 server is closing the connection because it has not been reused within a short enough period. Possibly within the period your code is using. Have you tried to upload the requests in parallel with a smaller delay between the uploads? This might help take advantage of the reused connection before S3 closes it.

[skyleelove]

Thanks a lot .
I set the /etc/sysctl.conf like this:

net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.core.somaxconn = 65535
net.ipv4.tcp_fin_timeout = 2

But it still has so many TIME_WAIT. Should I change my code to use the sdk to solve the problem with your suggestion.

[jasdel]

The theory behind the change I suggest is that by using a shorter delay period between upload your increase the probability of reusing a previous HTTP TCP connection.

Are you running into an issue where you're running out of sockets/handles that is related to the TIME_WAIT? If so I think it would be good to check to make sure that your code is closing all files that are being put to S3. If given an os.File for the io.ReadSeeker the SDK will not automatically close the file when the upload is complete.

If this isn't the issue you could try setting the aws#Config.S3ForcePathStyle configuration field to true. This will change the way your request is being made to S3 to not use the bucket in the domain name but in the URL path. This may help with the closed connections you're seeing.

[skyleelove]

Thanks a lot
I will try

[skyleelove]

Hi, I try it ,but it does not work.

[skyleelove]

Hi,jasdel

I find that every connection is disconnected by client,so which function of the sdk to break the connection.I can not find it.

[jasdel]

Hi @skyleelove, the SDK does not directly close the HTTP TCP connections. This is managed by the Go std libraries http.Client. if the connection is being closed, I think it is being closed server side.

We're you able to set the S3ForcePathStyle config value to see if that helped in this case?

[skyleelove]

Hi,jasdel
I got the way to solve this case.I rewrite this function in aws/config.go 246

this is old:

func (c *Config) WithHTTPClient(client *http.Client) *Config {
    c.HTTPClient = client
    return c
}

this is new that I change:

func (c *Config) WithHTTPClient(client *http.Client) *Config {
    ct := &http.Client{}
    ts := &http.Transport{
        Proxy: http.ProxyFromEnvironment,
        DialContext: (&net.Dialer{
                Timeout:   30 * time.Second,
                KeepAlive: 30 * time.Second,
        }).DialContext,
        MaxIdleConns:          100,
        IdleConnTimeout:       90 * time.Second,
        MaxIdleConnsPerHost:   100,
        TLSHandshakeTimeout:   3 * time.Second,
        ExpectContinueTimeout: 1 * time.Second,
    }
    ct.Transport = ts
    c.HTTPClient = ct
    return c
}

I set MaxIdleConnsPerHost to 100 in order to increase the connection number of same host to solve.And I want to kown the suggestion of you about this change.Thanks a lot.

[jasdel]

@skyleelove, Increasing the number of idle connections will improve the retention of connections in the pool. There looks to be a known issue (golang/go#13801) in Go's http.Client that causes it to more aggressively close connections as idle than really needed. The fix didn't make it into Go 1.7, but looks to be slated for 1.8.

How many requests at once is your application making?

To just specify a custom http.Client for the SDK I suggest passing it in to the SDK's config, when creating the service client. Something like the following will create a custom http.Client for the S3 service client to use when making requests.

sess := session.New()

svc := s3.New(sess, &aws.Config{HTTPClient: &http.Client{
    Transport: &http.Transport{
        Proxy: http.ProxyFromEnvironment,
        DialContext: (&net.Dialer{
            Timeout:   30 * time.Second,
            KeepAlive: 30 * time.Second,
        }).DialContext,
        MaxIdleConns:          100,
        IdleConnTimeout:       90 * time.Second,
        MaxIdleConnsPerHost:   100,
        TLSHandshakeTimeout:   3 * time.Second,
        ExpectContinueTimeout: 1 * time.Second,
    },
}})

The IdleConnTimeout probably doesn't need to be so long. In addition setting ExpectContinueTimeout to such a low value may cause timeout errors if the connection becomes congested or S3 doesn't respond with 100 continue soon enough.

[skyleelove]

Thanks a lot ,I will change the code.