Slim down neo package size

[devhawk]

The size of neo.dll and its dependencies when published (built on my local windows machine, release build, current master branch) is just over 28MB. Of this 28MB, over 20MB is SQLite native binaries and Entity Framework. Removing those brings the published size down to 6.2 MB. That's almost 80% of the package size for the legacy SQLite based UserWallet support.

I propose moving the SQLite wallet into a separate Neo.Wallets.SQLite assembly. This way, full nodes (neo-cli/neo-gui) can still have UserWallet support but so other managed tools in the Neo ecosystem (NCCS, Neo Express, Neo Debugger) can avoid this 5x bloat if they that don't need UserWallet support.

We could slim down further if we could remove the BouncyCastle dependency. That's about 2.3 MB (1/3 of the package size after removing SQLite & EF) and we only use it for secp256k1 support on MacOS and SCrypt support. I suspect we could find smaller, more focused implementations of secp256k1 and SCrypt that would allow use to remove the BouncyCastle dependency.

[devhawk]

Note, RpcServer.OpenWallet also depends on UserWallet, so either RpcServer project would take a dependency on the new Neo.Wallets.SQLite package. Alternatively, we could refactor the wallet JSON-RPC methods into a separate package as well so that the core RpcServer package could avoid the dependency

[devhawk]

Removing SQLite / EF dependency shrinks this package down to 40% of its original size.

• Neo.Compiler.CSharp package, Release build on my machine, master head: 17.6 MB
• Neo.Compiler.CSharp package, Release build on my machine, master head but compiled against a Neo.dll package w/o UserWallet: 7MB

Note, these are the compressed nuget package sizes, so they aren't directly comparable to the uncompressed publish folder sizes described above

[devhawk]

Looking thru the dependencies that remain after removing EF & SQLite, only bouncy castle stands out as a possible candidate for removal. The next biggest dependency after that is Akka (658kb) which also pulls in Newtonsoft.Json (678kb) and the remaining native dependencies (about 900kb). Obviously, removing Akka is not possible.

As an experiment, I removed the BouncyCastle dependency and commented out code that depends on it (MacOS secp256k1 & SCrypt) just to see how big it would be. Without BouncyCastle, SQLite and EF, release build on my machine the Neo.Compiler.CSharp package is down to about 6.2MB, or 35% of the original size.

Not sure the extra 800kb of compressed package size is worth it

[Jim8y]

#2564 #2547

[devhawk]

@Liaojinghui, I'm not sure why you listed these issues w/o additional context. I would very much still like to tackle #2547 at the earliest point possible. I originally pushed for something similar in #1865. I've never read #2564 before - seems like a reasonable thing to consider as part of a larger #2547/#1865 effort. Not sure it makes sense to tackle on it's own.

Regardless of these other issues, I believe we should still tackle this issue (and the associated PR #2679) for the 3.2 release because the EF + SQLite dependencies are very big and the legacy code that uses those dependencies is of such limited use.

[devhawk]

FYI @Liaojinghui I found a Neo branch on my dev box for where I was experimenting with a solution for #1865. It's VERY out of date (last updated Sept 2020) but you can take a look at the direction I was heading if you're interested: https://github.com/devhawk/neo/tree/devhawk/nucleus2.

I created a separate "neo.nucleus" package that had the core Crypto libraries, Neo's JSON stack, ISerializable + BinaryFormat, UInt160/256 and a bunch of the model classes (Block , Tx, etc). THe goal was to build a library that had only the parts that RpcClient and Neo.Compiler.CSharp depended on.

[Jim8y]

I just think what you are doing right now belongs to a part of something big, totally reconstruct the neo maybe.... So i put #2564 #2547 here to link them together as a reference

[devhawk]

I just think what you are doing right now belongs to a part of something big, totally reconstruct the neo maybe.... So i put #2564 #2547 here to link them together as a reference

The nucleus branch was some experimentation I did 18 months ago, so I'm not sure it qualifies as something I'm doing "right now".

While I support a longer-term effort to improve the layering of the core neo platform, I don't want to block #2679 because of it. Better layering in the long term is a fantastic goal, but #2679 significantly reduces package size for NCCS and the debugger right now.