Add development chain-spec file for minimal/parachain templates for Omni Node compatibility

[EleisonC]

Description

This PR adds development chain specs for the minimal and parachain templates. #6334

Integration

This PR adds development chain specs for the minimal and para chain template runtimes, ensuring synchronization with runtime code. It updates zombienet-omni-node.toml, zombinet.toml files to include valid chain spec paths, simplifying configuration for zombienet in the parachain and minimal template.

Review Notes

1. Overview of Changes:

• Added development chain specs for use in the minimal and parachain template.
• Updated zombienet-omni-node.toml and zombinet.toml files in the minimal and parachain templates to include paths to the new dev chain specs.

2. Integration Guidance:
   NB: Follow the templates' READMEs from the polkadot-SDK master branch. Please build the binaries and runtimes based on the polkadot-SDK master branch.

• Ensure you have set up your runtimes parachain-template-runtime and minimal-template-runtime
• Ensure you have installed the nodes required ie parachain-template-node and minimal-template-node
• Set up Zombinet
• For running the parachains, you will need to install the polkadot cargo install --path polkadot remember from the polkadot-SDK master branch.
• Inside the template folders minimal or parachain, run the command to start with Zombienet with Omni Node, Zombienet with minimal-template-node or Zombienet with parachain-template-node

Include your leftover TODOs, if any, here.

• Test the syncing of chain specs with runtime's code.

[cla-bot-2021]

User @EleisonC, please sign the CLA here.

[iulianbarbu]

Thanks @EleisonC ! Left a few comments for now.
Looking forward to the README updates & btw, have you clarified how to make a rust test for the chain specs?

[EleisonC]

Thanks @EleisonC ! Left a few comments for now. Looking forward to the README updates & btw, have you clarified how to make a rust test for the chain specs?

Hey @iulianbarbu that is what I am looking into today. Do you have any pointers you'd like me to know as I move forward?

[iulianbarbu]

Thanks @EleisonC ! Left a few comments for now. Looking forward to the README updates & btw, have you clarified how to make a rust test for the chain specs?

Hey @iulianbarbu that is what I am looking into today. Do you have any pointers you'd like me to know as I move forward?

From the top of my head I thought about adding an integration test. You could build a ChainSpec as described here: https://github.com/paritytech/polkadot-sdk/blob/master/templates/parachain/node/src/chain_spec.rs#L36, and then call as_json -> and then convert it to serde_json::Value, and then read the newly added file and load its content as a serde_json::Value and then compare the two.

[EleisonC]

Hey @iulianbarbu one last question it's okay if I placed the test folder/file under the nodes folder/file. if not is there a better place

[iulianbarbu]

Hey @iulianbarbu one last question it's okay if I placed the test folder/file under the nodes folder/file. if not is there a better place

Hmm, thought about placing it under runtime initially. I think it makes more sense to place it under runtime so that if anything changes here, running the tests for the crate would fail because the changes will not be inline with dev_chain_spec.json. The node uses the logic defined in the runtime, so it would benefit from the runtime test, but not directly. I don't think we need to add a test on the same lines in the templates' nodes crate either because these nodes aren't depending on the dev_chain_spec.json file. Only OmniNode depends on this file being in sync with the runtime, and that not because it would fail otherwise, but because it would start an outdated version.

[EleisonC]

From the top of my head I thought about adding an integration test. You could build a ChainSpec as described here: https://github.com/paritytech/polkadot-sdk/blob/master/templates/parachain/node/src/chain_spec.rs#L36, and then call as_json -> and then convert it to serde_json::Value, and then read the newly added file and load its content as a serde_json::Value and then compare the two.

Hey @iulianbarbu regarding comparing the two I am a bit stuck on what to compare. From my research I would think it's other the code or value of top
Is it the value of code? Because the test fails when I compare the code value of the two chain spec files.
let test_runtime_code = test_json_value // .get("genesis") // .expect("failed to extract genesis") // .get("runtimeGenesis") // .expect("failed to extract runtimeGenesis from genesis") // .get("code") // .expect("failed to extract code from runtimeGenesis");
But the code value of test_runtime and the existing file do not match. Hence wondering if it would be the Top value to or maybe I am missing a step of conversion somewhere

[EleisonC]

From the top of my head I thought about adding an integration test. You could build a ChainSpec as described here: https://github.com/paritytech/polkadot-sdk/blob/master/templates/parachain/node/src/chain_spec.rs#L36, and then call as_json -> and then convert it to serde_json::Value, and then read the newly added file and load its content as a serde_json::Value and then compare the two.

Hey @iulianbarbu regarding comparing the two I am a bit stuck on what to compare. From my research I would think it's other the code or value of top Is it the value of code? Because the test fails when I compare the code value of the two chain spec files. let test_runtime_code = test_json_value // .get("genesis") // .expect("failed to extract genesis") // .get("runtimeGenesis") // .expect("failed to extract runtimeGenesis from genesis") // .get("code") // .expect("failed to extract code from runtimeGenesis"); But the code value of test_runtime and the existing file do not match. Hence wondering if it would be the Top value to or maybe I am missing a step of conversion somewhere

I figured out my problem was around decoding a Hex String to bytes. I can now properly traverse the values of the Top section and find the code for that spec file

[iulianbarbu]

From the top of my head I thought about adding an integration test. You could build a ChainSpec as described here: https://github.com/paritytech/polkadot-sdk/blob/master/templates/parachain/node/src/chain_spec.rs#L36, and then call as_json -> and then convert it to serde_json::Value, and then read the newly added file an

d load its content as a serde_json::Value and then compare the two.

Hey @iulianbarbu regarding comparing the two I am a bit stuck on what to compare. From my research I would think it's other the code or value of top Is it the value of code? Because the test fails when I compare the code value of the two chain spec files. let test_runtime_code = test_json_value // .get("genesis") // .expect("failed to extract genesis") // .get("runtimeGenesis") // .expect("failed to extract runtimeGenesis from genesis") // .get("code") // .expect("failed to extract code from runtimeGenesis"); But the code value of test_runtime and the existing file do not match. Hence wondering if it would be the Top value to or maybe I am missing a step of conversion somewhere

I figured out my problem was around decoding a Hex String to bytes. I can now properly traverse the values of the Top section and find the code for that spec file

Happy to hear you've sorted it out! I've noticed that in some tests we don't compare the code portions of the generated chain specs, and that's mainly because they are not super relevant for the test case. For this test though I think that it is relevant, to ensure generated chain specs correspond to the runtimes in the CI.

Some other good news is that once we'll have these chain specs we'll be able to easily run in the CI the omni_node_* tests like this one: https://github.com/paritytech/polkadot-sdk/blob/master/templates/zombienet/tests/smoke.rs#L160. We can update them in a follow up, no need to consider them right now, but it is up to you.

As usual, LMK in case of blockers, happy to take a look.

[EleisonC]

Hey @iulianbarbu,
I added a test and would appreciate it if you could look at it and approve it before I add the one for the para chain. Additionally, I added the test folder to nodes because I was encountering setup issues.

[iulianbarbu]

Can you please regenerate the dev_chain_spec.json for minimal (for now) after building the minimal/runtime with cargo build? I wonder if you built the runtime with --release and then generated the dev_chain_spec.json. It makes sense according to the README, but we run the tests with the testnet profile, which enables the debug symbols, and I think the chain spec code section differs based on the profile. I would be curious to run the test again after you regenerate the dev_chain_spec.json, and also see if regenerating on my machine results in a different code section, or chain spec altogheter.

[skunert]

Looking good to me now! @iulianbarbu please verify the introduced workflow once more when you are back, then we are ready to go!

[skunert]

Here you are using matrix.package_name instead of matrix.template, I think this is a mismatch.

[EleisonC]

Thanks for the catch. I have addressed it

[iulianbarbu]

Just felt we can make the docs a bit clearer, so left a few clarifying comments, besides a few others for the misc-sync-templates.yml. It is mostly there :D

[EleisonC]

Hey @iulianbarbu done with the feedback.

[iulianbarbu]

Looks great! Thank you for sticking through this, it wasn't an easy one. :) I'll merge once you make the change for the README.docify.md.

[iulianbarbu]

Some trailing spaces in minimal/README.md make a required check fail. @EleisonC you can fix that and I will try merging after.

@alvicsam just as a headsup, you requested changes, you might want to take a look over this too before merging. It might be the case we can't merge without your approve as well. I'll keep this around until you're back actually, seeing you're away 'til Jan 26.

[EleisonC]

Some trailing spaces in minimal/README.md make a required check fail. @EleisonC you can fix that and I will try merging after.

Done