Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(plugins,fixtures,specs): Labeled fixture and execute formats #1220

Merged
merged 6 commits into from
Feb 18, 2025

Conversation

marioevz
Copy link
Member

@marioevz marioevz commented Feb 14, 2025

🗒️ Description

Labeled Fixture Formats in Spec Types

Adds labeled fixture and execute formats that can be defined by the spec tests for the fixtures they produce.

For example, state_test generated Blockchain and Blockchain engine fixtures now have the following labels:

  • LabeledFixtureFormat(BlockchainFixture, "blockchain_test_from_state_test")
  • LabeledFixtureFormat(BlockchainEngineFixture, "blockchain_test_engine_from_state_test")

This translates in the test IDs now showing "blockchain_test_from_state_test" and "blockchain_test_engine_from_state_test" instead of "blockchain_test" and "blockchain_test_engine" respectively.

These test cases now also have an extra marker that is the label.

  • blockchain_test_from_state_test and blockchain_test markers for Blockchain test.
  • blockchain_test_engine_from_state_test and blockchain_test_engine markers for Blockchain test.

This enables a bit more granularity when selecting the cases to fill (or execute in the case of execute), as we don't lose the -m blockchain_test, instead we now also can use -m blockchain_test_from_state_test to only fill blockchain tests that were generated from a State test.

I.e. -m blockchain_test still selects Blockchain tests generated from state tests, but we now also have -m blockchain_test_from_state_test which selects only Blockchain tests that were generated from a state test.

Remove StateTestOnlyand BlockchainTestEngine

Removes StateTestOnly and BlockchainTestEngine (and EOFTestOnly after #783) classes in favor of using markers to signal that a test should only be filled for a given fixture format:

This makes it easier to add the marker to a single parameter during parametrization, instead of having to write a new test function to single out tests that have this requirement.

Notes

Required for #783.

🔗 Related Issues

✅ Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
  • Tests: A PR with removal of converted JSON/YML tests from ethereum/tests have been opened.
  • Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

@marioevz marioevz added scope:pytest Scope: Changes EEST's pytest plugins type:feat type: Feature scope:fw Scope: Framework (evm|tools|forks|pytest) labels Feb 14, 2025
@marioevz marioevz requested a review from danceratopz February 14, 2025 19:58
@marioevz marioevz force-pushed the fixture-format-labels branch from f47fef2 to 0334a20 Compare February 17, 2025 17:56
@marioevz marioevz requested a review from spencer-tb February 17, 2025 18:01
Copy link
Collaborator

@spencer-tb spencer-tb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice PR. Easy to see what's going on. Only a few thoughts but nothing major!

Approved and LGTM (as I would be happy to merge as is)!

@spencer-tb
Copy link
Collaborator

Ohh and I'm being pedantic but could we add a changelog entry too 😅

@marioevz marioevz force-pushed the fixture-format-labels branch from 0334a20 to b2d3265 Compare February 18, 2025 00:12
@marioevz marioevz merged commit 21fb11c into main Feb 18, 2025
22 checks passed
@marioevz marioevz deleted the fixture-format-labels branch February 18, 2025 00:22
@marioevz marioevz restored the fixture-format-labels branch February 18, 2025 00:23
@marioevz marioevz deleted the fixture-format-labels branch February 18, 2025 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:fw Scope: Framework (evm|tools|forks|pytest) scope:pytest Scope: Changes EEST's pytest plugins type:feat type: Feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants