-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Execute benchmarks on aws #241
Conversation
Deployed test documentation to https://exasim-project.com/NeoFOAM/Build_PR_241 |
benchmarks/CMakeLists.txt
Outdated
if(WIN32) | ||
set_target_properties( | ||
bench_${TEST} | ||
PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/benchmarks/$<0:> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe add a comment why this is needed for windows? (especially the generator stuff).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed the generator expressions, I think they actually don't do anything atm.
648d987
to
4227948
Compare
This PR refactores our benchmark setup and aims to be a first step towards a continuos benchmarking approach.
It executes the benchmarks on aws, collects the results and pushes it as
json
to https://github.com/exasim-project/NeoFOAM-BenchmarkData for storage. All advanced post-processing etc. can be done on the NeoFOAM-BenchmarkData repo.In order for the current approach to work size and then executor need to be generated in a particular order see:
Additional Changes:
Limitations:
The benchmark output of Catch2 is very messy IMO and we need a single BENCHMARK statement per TEST_CASE.
Future work:
Once we merge this PR into main we can produce benchmark data on main to. Then we can execute the benchmark on aws twice, once for main once for the PR branch and compare the results. This allows to show the impact of a particular PR.