Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(specs): EOF: eof_test generates state tests, and execute in live devnets #783

Merged
merged 2 commits into from
Feb 19, 2025

Conversation

marioevz
Copy link
Member

@marioevz marioevz commented Sep 3, 2024

🗒️ Description

REQUIRES #1220 Merged

- eof_test produces state tests (and blockchain tests)

Create a state test out of an eof_test by:

  1. in the case of a run-time container, wrapping the container under test within an init container and sending a contract-creating transaction
  2. in the case of an init container, the container is wrapped in a factory container (which would execute the initcode to create a contract) and the factory container is wrapped within an init container, which is finally sent as a contract-creating transaction

- State test generated from EOF tests have extra markers

All state tests generated automatically from the eof_test spec now have a different test ID, along with an extra marker applied.

The test ID will show state_test_from_eof_test instead of simply state_test, e.g.:
test_swapn_on_max_stack[fork_Osaka-state_test_from_eof_test-swapn_operand_0]
intead of:
test_swapn_on_max_stack[fork_Osaka-state_test-swapn_operand_0]

Also, all tests will have the extra markers state_test_from_eof_test or blockchain_test_from_eof_test or blockchain_test_engine_from_eof_test in order to be able to fill only these specific tests.

E.g. using -m state_test will select all tests, including state_test_from_eof_test tests:

collected 28 items / 22 deselected / 6 selected

<Package eip663_dupn_swapn_exchange>
  <Module test_swapn.py>
    <Function test_swapn_all_valid_immediates[fork_Osaka-state_test]>
    <Function test_swapn_on_max_stack[fork_Osaka-state_test_from_eof_test-swapn_operand_0]>
    <Function test_swapn_on_max_stack[fork_Osaka-state_test_from_eof_test-swapn_operand_255]>
    <Function test_swapn_stack_underflow[fork_Osaka-state_test_from_eof_test-stack_height_0]>
    <Function test_swapn_stack_underflow[fork_Osaka-state_test_from_eof_test-stack_height_1]>
    <Function test_swapn_stack_underflow[fork_Osaka-state_test_from_eof_test-stack_height_21]>
    <Function test_swapn_stack_underflow[fork_Osaka-state_test_from_eof_test-stack_height_255]>

But using -m state_test_from_eof_test will only select state tests that have been automatically generated from an EOF test:

collected 28 items / 23 deselected / 5 selected

<Package eip663_dupn_swapn_exchange>
  <Module test_swapn.py>
    <Function test_swapn_on_max_stack[fork_Osaka-state_test_from_eof_test-swapn_operand_0]>
    <Function test_swapn_on_max_stack[fork_Osaka-state_test_from_eof_test-swapn_operand_255]>
    <Function test_swapn_stack_underflow[fork_Osaka-state_test_from_eof_test-stack_height_0]>
    <Function test_swapn_stack_underflow[fork_Osaka-state_test_from_eof_test-stack_height_1]>
    <Function test_swapn_stack_underflow[fork_Osaka-state_test_from_eof_test-stack_height_21]>
    <Function test_swapn_stack_underflow[fork_Osaka-state_test_from_eof_test-stack_height_255]>

Using -m eof_test will still produce the exact same tests as before.

- eof_test executable in live networks

This change allows the uv run execute command to send EOF initcode transactions to live networks by using the same methodology described in the previous section.

The marker transaction_post_from_eof_test will in this case send only EOF container validation tests to the live network:

collected 7 items / 2 deselected / 5 selected

<Package eip663_dupn_swapn_exchange>
  <Module test_swapn.py>
    <Function test_swapn_on_max_stack[fork_Osaka-transaction_post_from_eof_test-swapn_operand_0]>
    <Function test_swapn_on_max_stack[fork_Osaka-transaction_post_from_eof_test-swapn_operand_255]>
    <Function test_swapn_stack_underflow[fork_Osaka-transaction_post_from_eof_test-stack_height_0]>
    <Function test_swapn_stack_underflow[fork_Osaka-transaction_post_from_eof_test-stack_height_1]>
    <Function test_swapn_stack_underflow[fork_Osaka-transaction_post_from_eof_test-stack_height_21]>
    <Function test_swapn_stack_underflow[fork_Osaka-transaction_post_from_eof_test-stack_height_255]>

🔗 Related Issues

✅ Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
  • Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
  • Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

@marioevz marioevz force-pushed the eof-state-test-generation branch from 51b8181 to ed6f750 Compare September 20, 2024 21:18
@marioevz marioevz force-pushed the eof-state-test-generation branch from 5d28f2f to c13e837 Compare January 31, 2025 23:08
@marioevz marioevz changed the title feat(specs): EOF: eof_test generates state tests feat(specs): EOF: eof_test generates state tests, and execute in live devnets Jan 31, 2025
@marioevz marioevz force-pushed the eof-state-test-generation branch from 8fa0190 to b49c253 Compare February 5, 2025 16:11
@marioevz marioevz requested review from shemnon and chfast February 5, 2025 16:11
@marioevz marioevz marked this pull request as ready for review February 5, 2025 16:12
@marioevz marioevz force-pushed the eof-state-test-generation branch from 7dbed0f to b0e189d Compare February 5, 2025 21:58
@marioevz marioevz force-pushed the eof-state-test-generation branch from c066fe7 to c22bea9 Compare February 13, 2025 21:00
@marioevz marioevz force-pushed the eof-state-test-generation branch from c22bea9 to 005acab Compare February 14, 2025 20:08
@marioevz marioevz changed the base branch from main to fixture-format-labels February 14, 2025 20:08
@marioevz marioevz force-pushed the fixture-format-labels branch from f47fef2 to 0334a20 Compare February 17, 2025 17:56
@marioevz marioevz force-pushed the eof-state-test-generation branch 2 times, most recently from 3bec40e to 5e989aa Compare February 17, 2025 18:19
@marioevz marioevz force-pushed the fixture-format-labels branch from 0334a20 to b2d3265 Compare February 18, 2025 00:12
Base automatically changed from fixture-format-labels to main February 18, 2025 00:22
@marioevz marioevz force-pushed the eof-state-test-generation branch from 5e989aa to b4f6859 Compare February 18, 2025 00:26
@marioevz marioevz added type:feat type: Feature scope:fill Scope: fill command labels Feb 18, 2025
@marioevz marioevz force-pushed the eof-state-test-generation branch from 9b68305 to 193b29c Compare February 18, 2025 00:32
Copy link
Collaborator

@spencer-tb spencer-tb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Framework changes LGTM!

Copy link
Collaborator

@pdobacz pdobacz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just one call for expanding the docs a bit

@shemnon
Copy link
Collaborator

shemnon commented Feb 18, 2025

LGTM, modulo piotr's remark.

@marioevz marioevz force-pushed the eof-state-test-generation branch from 193b29c to f33221f Compare February 19, 2025 01:50
fix(specs): Reuse same initcontainer as deploy code for state tests gen from eof tests

fix(specs): Add docstrings for all `EOFTest` and `EOFStateTest` fields

fix(specs): docstring
fix(tests): Single tests where the deployed container could not be infered
@marioevz marioevz force-pushed the eof-state-test-generation branch from f33221f to fc6e701 Compare February 19, 2025 01:55
@marioevz
Copy link
Member Author

I changed the approach of the state tests generated for init-containers, and trying to infer the deployed container from the sections worked reasonably well, with just a single test that needed to be updated by setting its deployed_container field.

I've updated the method to be the default and will proceed to merge.

@marioevz marioevz merged commit b92eb56 into main Feb 19, 2025
21 checks passed
@marioevz marioevz deleted the eof-state-test-generation branch February 19, 2025 02:03
container-type section will be used. An error will be raised if the deployed container
cannot be inferred.

If the value is set to `None`, it is assumed that the container is invalid and the test will
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this actually accurate? not sth like:

If the value is set to None, it is assumed that the init container will fail to deploy, despite being valid EOF, and the test will expect that no contract is created.

?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:fill Scope: fill command type:feat type: Feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants