User cannot upload 30GB file

[dchoi27]

Don't have a ton of detail here but a user reached out to the support email with an issue uploading a large, 30GB file. Here's the errors. If anyone else encounters something similar please comment so we can work with you to understand what's going on.

### Tasks
- [ ] https://github.com/web3-storage/w3up/pull/1227
- [ ] https://github.com/web3-storage/fr32-sha2-256-trunc254-padded-binary-tree-multihash/pull/29
- [ ] release upload-client latest to console

[vasco-santos]

@dchoi27 by the stack trace this is something not working with Piece computation. I also think other issue we saw from Ian hits the same code path in the WASM implementation of the Piece hashing

[Gozala]

There seems to be multiple issues here at play. Looking into this I discovered that we keep using increasingly more memory and at around 4GB wasm starts to fail.

One issue is it looks like we fail do dealocate memory tracked by storacha/fr32-sha2-256-trunc254-padded-binary-tree-multihash#28

But even if I remove the Piece hashing entirely here

https://github.com/web3-storage/w3up/blob/153da70e06e9146783193e81cd85294eafdc17e8/packages/upload-client/src/index.js#L133-L138

We still keep using more and more memory around 15GB when uploading 17GB file and we climb up really quickly around when only 6% is processed

If I comment following lines out, memory use is under 200MB so there is something wrong going on within the commented block

https://github.com/web3-storage/w3up/blob/153da70e06e9146783193e81cd85294eafdc17e8/packages/upload-client/src/store.js#L58-L133

[Gozala]

I'm focusing on storacha/fr32-sha2-256-trunc254-padded-binary-tree-multihash#28

@gobengo if you'll have time maybe you can take a look at the other leak in the store.js file

[gobengo]

I'm hopeful that the root cause of this was #1227 which has been released in upload-client.
@travis will deploy to console sometime today and it should include that new upload-client, and then I'll test this again. (I tested a bit ago with a 6GB file and I reproduced the error still)

[gobengo]

I was able to upload a 6GB file just now even though it used to trigger this same error. I am hopeful 🤞🏻 this might be fixed. @dchoi27 could you ask the end-user to retest?

[dchoi27]

Ah unfortunately they asked to have their account deleted when this didn't work. Maybe if we get 1-2 other points of validation we can close this.

[dchoi27]

ok - the original user got back to me and confirmed it worked! closing this