update ethereumjs libraries

[mpetrunic]

Description

ethereumjs/common received big update which reduces bundle size significantly, unfortunately:

• ethereumjs/tx now depends on ethers - tx: reduce number of ethers dependencies ethereumjs/ethereumjs-monorepo#2513 should mitigate this slightly once released
• ethereumjs/util depends on async which breaks our webpack build - proposed PR feat: remove async library ethereumjs/ethereumjs-monorepo#2514 to remove async library altogether (if that doesn't work, will try to propose to just use import ... from "async/eachSeries")

closes #5943

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist for 1.x:

• I have selected the correct base branch.
• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings.
• Any dependent changes have been merged and published in downstream modules.
• I ran npm run dtslint with success and extended the tests and types if necessary.
• I ran npm run test:cov and my test cases cover all the lines and branches of the added code.
• I ran npm run build with success.
• I have tested the built dist/web3.min.js in a browser.
• I have tested my code on the live network.
• I have checked the Deploy Preview and it looks correct.
• I have updated the CHANGELOG.md file in the root folder.

Checklist for 4.x:

• I have selected the correct 4.x base branch.
• Within the description, the feature or issue is discussed in detail or an issue is linked.
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas.
• I have added any required tests for the changes I made
• I have made corresponding changes to the documentation
• Any dependent changes have been merged and published in downstream modules.
• I ran yarn successfully
• I ran yarn lint successfully
• I ran yarn build:web successfully
• I ran yarn test:unit successfully
• I ran yarn test:integration successfully
• I ran compile:contracts successfully
• I have tested my code.
• I have updated the corresponding CHANGELOG.md file in the packages I have edited.

[mpetrunic]

Heck, seems like with latest ethereumjs libs we get ssz and ethers providers libraries which are still large :(

[github-actions]

Bundle Stats

Hey there, this message comes from a github action that helps you and reviewers to understand how these changes affect the size of this project's bundle.

As this PR is updated, I'll keep you updated on how the bundle size is impacted.

Total

Asset	Old size	New size	Diff	Diff %
Total	1.06 MB	1.37 MB	310 KB	28.49%

View detailed bundle breakdown

Added

No assets were added

Removed

No assets were removed

Bigger

Asset	Old size	New size	Diff	Diff %
web3.min.js	1.05 MB	1.35 MB	310 KB	28.84%

Smaller

No assets were smaller

Unchanged

Asset	Old size	New size	Diff	Diff %
../lib/accounts.d.ts	3.42 KB	3.42 KB	0	0.00%
../lib/types.d.ts	1.68 KB	1.68 KB	0	0.00%
../lib/types.d.ts.map	1.51 KB	1.51 KB	0	0.00%
../lib/abi.d.ts	999 bytes	999 bytes	0	0.00%
../lib/index.d.ts.map	919 bytes	919 bytes	0	0.00%
../lib/index.d.ts	899 bytes	899 bytes	0	0.00%
../lib/web3.d.ts	842 bytes	842 bytes	0	0.00%
../lib/web3.d.ts.map	694 bytes	694 bytes	0	0.00%
../lib/accounts.d.ts.map	528 bytes	528 bytes	0	0.00%
../lib/eth.exports.d.ts.map	358 bytes	358 bytes	0	0.00%
../lib/eth.exports.d.ts	321 bytes	321 bytes	0	0.00%
../lib/providers.exports.d.ts.map	292 bytes	292 bytes	0	0.00%
../lib/providers.exports.d.ts	238 bytes	238 bytes	0	0.00%
../lib/version.d.ts.map	140 bytes	140 bytes	0	0.00%
../lib/abi.d.ts.map	124 bytes	124 bytes	0	0.00%
../lib/version.d.ts	97 bytes	97 bytes	0	0.00%

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Pages

Latest commit:	bfad5e0
Status:	✅  Deploy successful!
Preview URL:	https://0bdf64e9.web3-js-docs.pages.dev
Branch Preview URL:	https://chore-update-ethereumjs-libr.web3-js-docs.pages.dev

View logs

[mpetrunic]

Heck, seems like with latest ethereumjs libs we get ssz and ethers providers libraries which are still large :(

Heck, seems like with latest ethereumjs libs we get ssz and ethers providers libraries which are still large :(

cc @web3/web3-js

[jdevcs]

This PR can be closed , as eth/tx functions moved into 4.x

[holgerd77]

Hi there, Holger from EthereumJS, we have a fix in the works for the Ethers dependency (with the goal to remove): ethereumjs/ethereumjs-monorepo#2633, feedback would be appreciated! 🙂

SSZ we still need to discuss.

[holgerd77]

(respectively, to manage expectations: it is more unlikely that we will remove SSZ, since this is becoming such an integral part of the protocol in the near distant future. Might be a way here to continue to work together to make the SSZ library more compatible (if needed), lessen dependencies, size,...)

[holgerd77]

(might generally be worth to have monthly ChainSafe(so mainly: Lodestar, Web3.js)/EthereumJS calls to discuss things like that)

[jdevcs]

@holgerd77 For optimising 4.x web3.js build size we have removed some ethereumjs dependencies by using its functionality #5963

[jdevcs]

(might generally be worth to have monthly ChainSafe(so mainly: Lodestar, Web3.js)/EthereumJS calls to discuss things like that)

cc: @mconnelly8

[holgerd77]

@holgerd77 For optimising 4.x web3.js build size we have removed some ethereumjs dependencies by using its functionality #5963

To be honest: I think that is totally a dead end, especially our Common library is evolving so quickly that you will likely (and unnecessarily) waste a lot of ressources to catch up.

Guess it would be beneficial to find some way together here. If you have some additional demands, just open an issue on our side https://github.com/ethereumjs/ethereumjs-monorepo or mention people from our team. 🙂

[mpetrunic]

@holgerd77 For optimising 4.x web3.js build size we have removed some ethereumjs dependencies by using its functionality #5963

To be honest: I think that is totally a dead end, especially our Common library is evolving so quickly that you will likely (and unnecessarily) waste a lot of ressources to catch up.

Guess it would be beneficial to find some way together here. If you have some additional demands, just open an issue on our side https://github.com/ethereumjs/ethereumjs-monorepo or mention people from our team. slightly_smiling_face

Hey @holgerd77,
this change reduced web3.js bundle size by ~30%.
Some of the stuff that we would need before we could consider using it again:

• replace Buffer with Uint8Array or use Buffer polyfill explicitly (we would prefer the first one as we already modified the code to use Uint8Array 6004 replace buffer with uint8array #6033). Your code expects that users will provide Buffer implementation in the browser which is very confusing for them.
• change typescript config so "imports" are not transpiled into "require's"
• run some basic checks to ensure common and tx packages work in browser
• making transaction builder modular
  • so we can decide which transaction types you are supporting. Like, I don't think we will add eip4844 tx by default but rather as an optional plugin since those tx types add a lot of weight to the bundle but it's not gonna be used much by regular users
  • allowing us to add different transaction types ourselves or through plugins

Let me know if you think I should open those issues on ethereumjs repo or if you don't think this aligns with ethereumjs roadmap^^

[paulmillr]

@mpetrunic do you use tree-shaking? Does it work?

replace Buffer with Uint8Array

This is planned for v7 and huge amount of work for this was done over the last couple months.

making transaction builder modular

That seems like it should be solved with tree-shaking.

[mpetrunic]

@mpetrunic do you use tree-shaking? Does it work?

Yes we use tree shaking when creating minified bundle.

making transaction builder modular

That seems like it should be solved with tree-shaking.

Treeshaking cannot remove dynamic code from TransactionFactory so we end up with ssz and ckzg code from eip4844 transaction type. I feel like better approach would be to allow users to instantiate transaction factory with transaction types they want to support^^

TBH, we could've fixed that if we just wrote our TxFactory that uses individual tx implementations.

We can promise to try library again with v7 version :)