aws-s3-multipart uploads without companion sometimes fail for large size files

[Saccarab]

I've written s3/multipart endpoints to my backend express app as it was described here #1189 (comment) However for longer file uploads (often above 6 or 7GBs) I have some reports of failed upload attempts.

[Uppy] [21:15:15] Failed to upload file.mov Could not get https://[BACKEND_URL]/s3/multipart/MjI48PCRt2X3ZsqRjYWLUzZVXAC4a_UCXhrlvX9X0lIjbwdf5vFlFAE5mF4wpX6q_fixqSY8E2cCa2e44bmDGQ--/506?key=rnMYcJrHeqPD7FivUKGfH9Hesku2%2FnagHvPitLjPm1UF2fWhyI.mov. Error: This looks like a network error, the endpoint might be blocked by an internet provider or a firewall. Source error: [TypeError: Failed to fetch]

This is the error template I got off my error reporting tool which only fires for 'delete' and 'get' requests.

I assume this not be a CORS issue since I am able to upload successfully on my network.

I went through this issue and set the simultaneous chunk upload limit to 5.

this.uppy.use(AwsS3Multipart, { limit: 5, companionUrl: '[BACKEND_URL]' })

Also tried, setting parser limit to 50 MBs as It was discussed on the post but the issue persists.

app.use(express.json({limit: '50mb'}));
app.use(express.urlencoded({limit: '50mb', extended: true}));

Could this be the client timing out after some time for longer files or is it more likely to be related to the server side?

[goto-bus-stop]

Do you know if it's limited to a particular browser? I've heard of this happening somewhat randomly on certain Safari versions. I don't know what causes it, but #2312 should increase the S3 Multipart plugin's robustness in general and will hopefully work around this problem as well.

[Saccarab]

I have a lot of reports for the same exact event specifically on get and delete s3/multipart requests, but as far as I can tell it doesn't look browser-related at this point.

[goto-bus-stop]

Hmm. do you have any insight into why the requests are failing? Perhaps from server-side logging? If I had to guess the requests are failing somewhere on the server side, and the error path is not setting CORS headers, so the failures are indistinguishable from a network error on the client side. (It's possible the original failure is still because of an Uppy bug, if it's sending bogus data for some reason.)

[Saccarab]

Unfortunately, I don't have any hints on why requests could be failing on my server-side, I've assumed it not to be related to CORS since the issue is exclusive to bigger sized files and I am pretty much not able to reproduce the issue on my environment. I, however, can reproduce it on a coworker's environment so I can try to get some useful information off of that.

[Velora]

We may be experiencing a similar issue. Specifically for larger file sizes, though in our case larger is as small as 66MB and as high as 2GB. We have a user that can reproduce the issue 100% of the time in any browser on their Windows 10 machine with a 66mb file. It seems that the file is uploading to our cache store on S3, but the user is never getting a success event returned.

I posted some additional details in #1699

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kode-ninja]

This issue still happens on production to our users, all the time.

I've looked at #1699 too and I must say that it just happened yesterday on a Mac/Safari Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.2 Safari/605.1.15.

We couldn't find anything in common to all these failures, only that it doesn't happen when uploading relatively small files.

Our timeout is configured to 60 seconds (instead of the default 30).

Unfortunately, we can't seem to reproduce this in-house.

Any recommendations?
Is there any debugging information that is needed and may be helpful? We may be able to collect it on production and help figuring this one out..