[3.2] Implement JSON Snapshot Reader #25
Conversation
…erge snapshot and jsonsnapshot code handling
…d of adding a new option. Improved memory usage so current WAX snapshot can be supported. Modified the JSON output so that table rows are an array which reduces file size and memory usage. Put rapidjson inside an eosio_rapidjson namespace to avoid ODR issues. Change CMakeLists to not compile rapidjson but only include it as it is a header only library.
…ent possible collision. Rename rapidjson in libraries.
|
Is there any reasonable way to sniff the file and determine binary vs JSON, instead of relying on the file name? |
Maybe, is that worth doing? The |
Maybe. Being prescriptive about filenames can break more flexible invocations. Like maybe someone could set up a pipeline to modify a snapshot without ever writing it to disk via something like
But this could just be academic. The binary snapshots need to be seekable at the moment; I'm not sure if the JSON ones seek around or not, and it may be hard to accommodate something like the above command even for JSON, idk |
| @@ -5,6 +5,7 @@ | |||
| #include <fc/variant_object.hpp> | |||
| #include <boost/core/demangle.hpp> | |||
| #include <ostream> | |||
| #include <memory> | |||
|
|
|||
| namespace eosio { namespace chain { | |||
There was a problem hiding this comment.
Please rebase this PR to main (or start a new PR) and incorporate libraries/chain/include/eosio/chain/snapshot.hpp & libraries/chain/snapshot.cpp changes from #24 into it. Then we will be able to get the PR into main. The --snapshot-to-json portion of #24 can be added to leap-util later (keep that PR up so it can be easily incorporated into leap-util.
|
[@spoonincode sorry, somehow edited your comment instead of quoting it. I didn't even think that was possible.]
We could just try reading as binary which should quickly fail as it will not find the magic number of the binary file, then switch over to JSON. |
eosnetworkfoundation/mandel#710 ports a way to convert a snapshot to json. This PR enhances the --snapshot option to support reading a snapshot in JSON format if the snapshot has .json extension.
Resolves: eosnetworkfoundation/mandel#771
Previous discussion: eosnetworkfoundation/mandel#732