-
Notifications
You must be signed in to change notification settings - Fork 369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a hook so that an AmrLevel can force a post-timestep regrid #1
Conversation
The motivation for this change is that sometimes, during the course of a coarse timestep, the timestep will found to be unstable due to a lack of resolution. In this case the coarse level may want to signal that more resolution is required immediately (as opposed to waiting for a future regrid), because even a single unstable step may cause significant harm to the integrity of the simulation. An example is a spurious numerical detonation caused by insufficient resolution in a thermonuclear burning region. If this numerically-seeded detonation is detected at the end of a timestep, more levels can be added immediately so that the fine levels can improve the situation and quench the detonation (since they will overwrite the coarse data at the end of the timestep). Each AmrLevel now has a member variable post_step_regrid. To trigger a regrid at the end of a step, but before any fine timesteps would be taken, they only need to set this variable to one. At the beginning of every timestep, it will be reset to 0.
Should What happens to the coarse fluxes stored the flux register that is a member of the fine level? |
I think not. This should be considered a 'special' regrid that does not interfere with the normal regrid process. This will lead sometimes lead to back-to-back regrids (one at the end of a step and then another at the beginning of the next) but this is fine with me; those regrids are not necessarily redundant, and in any case it's a price the user is choosing to pay by opting in to this flexibility. |
I might have misunderstood this. Each AmrLevel has a post_step_regrid flag. It looks like it indicates whether or not the fine levels should have new grids, because regrid doesn't change the base level. But if the fine levels change, shouldn't the current level be run again so that FluxRegister can be set properly. Or do you simply save all the currently fluxes? |
I originally started with the idea of running the current level again, but I realized that it is a big win to avoid re-doing the step if it is possible. So for this implementation, I think that saving the fluxes is a requirement. At the beginning of an advance, the fine level should check whether the level below it has triggered post_step_regrid. If so, it should zero out the coarse fluxes and re-fill them. |
Add missing file for 1D convergence test
* CMake: first basic attempt at HIP support * Update Tools/CMake/AMReXParallelBackends.cmake Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> * CI: Add HIP-Clang * fix typo: GCC 9.3 on focal * Fix sources: set two lists * fix target properties: append two lists * CMake: some changes to debug issue with generated files * CI: only run HIP check for now * CMake: fix typo * CMake: let's try this * CMake: other small modifications * CMake: attemp #1 * CMake: attemp #2 * CI: build just the bare minimum for now * CMake: attempt #3 * CMake: missing defines for HIP builds * CMake: defines were still missing :-P * CMake: AMREX_HIP_PLATFORM is actually the HIP_COMPILER * CMake: let's see if this works * Install rocRAND * CMake: re-factor and update defines for HIP compilation * CMake: list against hiprand too * CMake: find and link to hiprand * CMake: add -DNDEBUG to any type of build when HIP is on * CMake: we must manually add rocrand too * CMake: let's see if it works with Fortran enabled * Revert "CMake: let's see if it works with Fortran enabled" This reverts commit 0650715. * CMake: try this * CMake: this should work too * Revert "Revert "CMake: let's see if it works with Fortran enabled"" This reverts commit 6a92adc. * CMake: no fortran for now * CI: turn on linear solvers in HIP check * CI: turn on particles in HIP checks * Revert "CI: turn on linear solvers in HIP check" This reverts commit c723560. The reason is that Linear Solvers tutorials are not HIP-aware yet. * CMake: hipify tutorials * CMake: temporary fix * Revert "CMake: temporary fix" This reverts commit bd6724e. * CMake: fix HIP compilation for Tutorials * CMake: some cleanup * CMake: add HIP options to pass in architecture and extra flags * CMake: oops * CMake: try to enable linear solvers * CMake: update ROCm version number in CI check name * CMake: enable Fortran * CMake: disable tutorials for the time being * CMake: commit custom FindHIP.cmake to debug the issue * CMake: trying this * CMake: now it should use the local FindHIP.cmake * CMake: now custom FindHIP.cmake should be able to find helper files * CMake: let's see if we really need this * CMake: let's try this * CMake: temporary fix * Revert "CMake: disable tutorials for the time being" This reverts commit 885b911. * CMake: Tutorials/Particles/CellSortedParticles do not work with HIP * CMake: remove local FindHIP.cmake. * CMake: HIP arch flags are now PUBLIC and inheritable * CMake: no use for setup_target_for_hip_compilation anymore * CMake: remove unnecessary options for HIP * CMake: check that HIP_COMPILER is the same as CMAKE_CXX_COMPILER * CMake: better this * CI: uncomment all checks * CMake: fix visibility of arch flags for HIP * Doc: add subsection on HIP+CMake * Update Docs/sphinx_documentation/source/GPU.rst Co-authored-by: Shreyas Ananthan <shreyas@umd.edu> * CMake: fix incorrect configuration option passed to cmake * CI: target 'tutorials' no longer exists Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja> Co-authored-by: Shreyas Ananthan <shreyas@umd.edu>
…rena GNUmake and DPC++ updates
It can turn "N5amrex3BoxE" into "amrex::Box", and "Z4mainEUlvE_" into "main::{lambda()#1}".
The motivation for this change is that sometimes, during the course of a coarse
timestep, the timestep will found to be unstable due to a lack of resolution. In
this case the coarse level may want to signal that more resolution is required
immediately (as opposed to waiting for a future regrid), because even a single
unstable step may cause significant harm to the integrity of the simulation. An
example is a spurious numerical detonation caused by insufficient resolution in
a thermonuclear burning region. If this numerically-seeded detonation is detected
at the end of a timestep, more levels can be added immediately so that the fine
levels can improve the situation and quench the detonation (since they will
overwrite the coarse data at the end of the timestep).
Each AmrLevel now has a member variable post_step_regrid. To trigger a regrid
at the end of a step, but before any fine timesteps would be taken, they only need
to set this variable to one. At the beginning of every timestep, it will be reset to 0.