Content-Length: 360622 | pFad | http://github.com/ethereum/execution-spec-tests/pull/1701

10 feat(tests): add more worst-case modexp benchmarks by chfast · Pull Request #1701 · ethereum/execution-spec-tests · GitHub
Skip to content

feat(tests): add more worst-case modexp benchmarks #1701

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat(tests): add more worst-case modexp benchmarks #1701

wants to merge 1 commit into from

Conversation

chfast
Copy link
Member

@chfast chfast commented Jun 2, 2025 •
edited
Loading

🗒️ Description

This refactors the test_worst_modexp() with parametrized precompile inputs. It adds inputs consting ~500 gas for the EIP-7883 review.

🔗 Related Issues

✅ Checklist

  • All: Set appropriate labels for the changes.
  • All: Considered squashing commits to improve commit history.
  • All: Added an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • Tests: All converted JSON/YML tests from ethereum/tests/tests/static have been assigned @ported_from marker.
  • Tests: A PR with removal of converted JSON/YML blockchain tests from ethereum/tests have been opened.
  • Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.

@chfast chfast requested a review from jsign June 2, 2025 15:37
@chfast chfast added scope:tests Scope: Changes EL client test cases in `./tests` type:refactor Type: Refactor type:feat type: Feature feature:zkevm labels Jun 2, 2025
@chfast
Copy link
Member Author

chfast commented Jun 2, 2025 •
edited
Loading

Benchmark results

Geth

evm statetest --bench fixtures/state_tests/zkevm/worst_compute/worst_modexp.json
{"stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b"}
{"stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b"}
{"stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b"}
{"stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b"}
[
  {
    "name": "tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_24b]",
    "pass": true,
    "stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b",
    "fork": "Osaka",
    "benchStats": {
      "time": 1557245740,
      "allocs": 6412901,
      "bytesAllocated": 352983712,
      "gasUsed": 72000000
    }
  },
  {
    "name": "tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_32b]",
    "pass": true,
    "stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b",
    "fork": "Osaka",
    "benchStats": {
      "time": 1056881439,
      "allocs": 2783521,
      "bytesAllocated": 169499480,
      "gasUsed": 72000000
    }
  },
  {
    "name": "tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_8b]",
    "pass": true,
    "stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b",
    "fork": "Osaka",
    "benchStats": {
      "time": 2286681115,
      "allocs": 6556145,
      "bytesAllocated": 261662376,
      "gasUsed": 72000000
    }
  },
  {
    "name": "tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_16b]",
    "pass": true,
    "stateRoot": "0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b",
    "fork": "Osaka",
    "benchStats": {
      "time": 1623244585,
      "allocs": 6907389,
      "bytesAllocated": 323491848,
      "gasUsed": 72000000
    }
  }
]

Besu

besu-evm state-test fixtures/state_tests/zkevm/worst_compute/worst_modexp.json
{"output":"","gasUsed":"0x44aa200","time":1530777000,"Mgps":"47.035","test":"tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_16b]","fork":"Osaka","d":0,"g":0,"v":0,"stateRoot":"0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":true,"error":"Out of gas"}
{"output":"","gasUsed":"0x44aa200","time":1076826660,"Mgps":"66.863","test":"tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_24b]","fork":"Osaka","d":0,"g":0,"v":0,"stateRoot":"0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":true,"error":"Out of gas"}
{"output":"","gasUsed":"0x44aa200","time":678852354,"Mgps":"106.061","test":"tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_32b]","fork":"Osaka","d":0,"g":0,"v":0,"stateRoot":"0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":true,"error":"Out of gas"}
{"output":"","gasUsed":"0x44aa200","time":1807314427,"Mgps":"39.838","test":"tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Osaka-state_test-worst_8b]","fork":"Osaka","d":0,"g":0,"v":0,"stateRoot":"0x1d3c59311b602ed6cdbb8766efb327a88c7764cb788f2ffbe9deef6b8d61ff7b","postLogsHash":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347","pass":true,"error":"Out of gas"}

@marioevz
Copy link
Member

marioevz commented Jun 3, 2025

This PR might be in conflict with this commit d9da7de from #1579

@chfast
Copy link
Member Author

chfast commented Jun 3, 2025

This PR might be in conflict with this commit d9da7de from #1579

I will wait.

@chfast chfast marked this pull request as draft June 3, 2025 12:25
@chfast chfast mentioned this pull request Jun 4, 2025
Draft
@marioevz
Copy link
Member

marioevz commented Jun 4, 2025

#1579 is merged, can now rebase to main and check if everything is still working.

@chfast
feat(tests): add more worst-case modexp benchmarks
0c3ce41 
This refactors the `test_worst_modexp()` with parametrized precompile
inputs. It adds inputs consting ~500 gas for the EIP-7883 review.
@chfast chfast marked this pull request as ready for review June 4, 2025 16:40
jsign
jsign approved these changes Jun 6, 2025
View reviewed changes
Copy link
Collaborator

@jsign jsign left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_32b_exp_40]-1 22858304416
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_32b_exp_96]-1 26373254772
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_32b_exp_256]-1        28089552372
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_24b_exp_168]-1        29398879265
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_16b_exp_320]-1        36480039911
tests/zkevm/test_worst_compute.py::test_worst_modexp[fork_Cancun-blockchain_test_from_state_test-mod_even_8b_exp_896]-1 65149073029

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jsign jsign jsign approved these changes

@marioevz marioevz marioevz approved these changes

Assignees
No one assigned
Labels
feature:zkevm scope:tests Scope: Changes EL client test cases in `./tests` type:feat type: Feature type:refactor Type: Refactor
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@chfast @marioevz @jsign








ApplySandwichStrip

pFad - (p)hone/(F)rame/(a)nonymizer/(d)eclutterfier!      Saves Data!


--- a PPN by Garber Painting Akron. With Image Size Reduction included!

Fetched URL: http://github.com/ethereum/execution-spec-tests/pull/1701

Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy