Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a hook so that an AmrLevel can force a post-timestep regrid #1

Merged
merged 1 commit into from
May 7, 2017

Conversation

maximumcats
Copy link
Member

The motivation for this change is that sometimes, during the course of a coarse
timestep, the timestep will found to be unstable due to a lack of resolution. In
this case the coarse level may want to signal that more resolution is required
immediately (as opposed to waiting for a future regrid), because even a single
unstable step may cause significant harm to the integrity of the simulation. An
example is a spurious numerical detonation caused by insufficient resolution in
a thermonuclear burning region. If this numerically-seeded detonation is detected
at the end of a timestep, more levels can be added immediately so that the fine
levels can improve the situation and quench the detonation (since they will
overwrite the coarse data at the end of the timestep).

Each AmrLevel now has a member variable post_step_regrid. To trigger a regrid
at the end of a step, but before any fine timesteps would be taken, they only need
to set this variable to one. At the beginning of every timestep, it will be reset to 0.

The motivation for this change is that sometimes, during the course of a coarse
timestep, the timestep will found to be unstable due to a lack of resolution. In
this case the coarse level may want to signal that more resolution is required
immediately (as opposed to waiting for a future regrid), because even a single
unstable step may cause significant harm to the integrity of the simulation. An
example is a spurious numerical detonation caused by insufficient resolution in
a thermonuclear burning region. If this numerically-seeded detonation is detected
at the end of a timestep, more levels can be added immediately so that the fine
levels can improve the situation and quench the detonation (since they will
overwrite the coarse data at the end of the timestep).

Each AmrLevel now has a member variable post_step_regrid. To trigger a regrid
at the end of a step, but before any fine timesteps would be taken, they only need
to set this variable to one. At the beginning of every timestep, it will be reset to 0.
@maximumcats maximumcats requested a review from WeiqunZhang May 1, 2017 15:19
@maximumcats maximumcats changed the base branch from master to development May 1, 2017 15:19
@WeiqunZhang
Copy link
Member

Should level_count be reset for the new levels?

What happens to the coarse fluxes stored the flux register that is a member of the fine level?

@maximumcats
Copy link
Member Author

Should level_count be reset for the new levels?

I think not. This should be considered a 'special' regrid that does not interfere with the normal regrid process. This will lead sometimes lead to back-to-back regrids (one at the end of a step and then another at the beginning of the next) but this is fine with me; those regrids are not necessarily redundant, and in any case it's a price the user is choosing to pay by opting in to this flexibility.

@WeiqunZhang
Copy link
Member

I might have misunderstood this. Each AmrLevel has a post_step_regrid flag. It looks like it indicates whether or not the fine levels should have new grids, because regrid doesn't change the base level. But if the fine levels change, shouldn't the current level be run again so that FluxRegister can be set properly. Or do you simply save all the currently fluxes?

@maximumcats
Copy link
Member Author

I originally started with the idea of running the current level again, but I realized that it is a big win to avoid re-doing the step if it is possible. So for this implementation, I think that saving the fluxes is a requirement. At the beginning of an advance, the fine level should check whether the level below it has triggered post_step_regrid. If so, it should zero out the coarse fluxes and re-fill them.

@WeiqunZhang WeiqunZhang merged commit d29d0f4 into development May 7, 2017
@maximumcats maximumcats deleted the maxpkatz/post_timestep_regrid branch May 17, 2017 15:30
atmyers pushed a commit that referenced this pull request Nov 30, 2017
Add missing file for 1D convergence test
bcfriesen pushed a commit that referenced this pull request Nov 6, 2019
mic84 added a commit to mic84/amrex that referenced this pull request Sep 16, 2020
mic84 added a commit that referenced this pull request Oct 13, 2020
* CMake: first basic attempt at HIP support

* Update Tools/CMake/AMReXParallelBackends.cmake

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>

* CI: Add HIP-Clang

* fix typo: GCC 9.3 on focal

* Fix sources: set two lists

* fix target properties: append two lists

* CMake: some changes to debug issue with generated files

* CI: only run HIP check for now

* CMake: fix typo

* CMake: let's try this

* CMake: other small modifications

* CMake: attemp #1

* CMake: attemp #2

* CI: build just the bare minimum for now

* CMake: attempt #3

* CMake: missing defines for HIP builds

* CMake: defines were still missing :-P

* CMake: AMREX_HIP_PLATFORM is actually the HIP_COMPILER

* CMake: let's see if this works

* Install rocRAND

* CMake: re-factor and update defines for HIP compilation

* CMake: list against hiprand too

* CMake: find and link to hiprand

* CMake: add -DNDEBUG to any type of build when HIP is on

* CMake: we must manually add rocrand too

* CMake: let's see if it works with Fortran enabled

* Revert "CMake: let's see if it works with Fortran enabled"

This reverts commit 0650715.

* CMake: try this

* CMake: this should work too

* Revert "Revert "CMake: let's see if it works with Fortran enabled""

This reverts commit 6a92adc.

* CMake: no fortran for now

* CI: turn on linear solvers in HIP check

* CI: turn on particles in HIP checks

* Revert "CI: turn on linear solvers in HIP check"

This reverts commit c723560.
The reason is that Linear Solvers tutorials are not HIP-aware yet.

* CMake: hipify tutorials

* CMake: temporary fix

* Revert "CMake: temporary fix"

This reverts commit bd6724e.

* CMake: fix HIP compilation for Tutorials

* CMake: some cleanup

* CMake: add HIP options to pass in architecture and extra flags

* CMake: oops

* CMake: try to enable linear solvers

* CMake: update ROCm version number in CI check name

* CMake: enable Fortran

* CMake: disable tutorials for the time being

* CMake: commit custom FindHIP.cmake to debug the issue

* CMake: trying this

* CMake: now it should use the local FindHIP.cmake

* CMake: now custom FindHIP.cmake should be able to find helper files

* CMake: let's see if we really need this

* CMake: let's try this

* CMake: temporary fix

* Revert "CMake: disable tutorials for the time being"

This reverts commit 885b911.

* CMake: Tutorials/Particles/CellSortedParticles do not work with HIP

* CMake: remove local FindHIP.cmake.

* CMake: HIP arch flags are now PUBLIC and inheritable

* CMake: no use for setup_target_for_hip_compilation anymore

* CMake: remove unnecessary options for HIP

* CMake: check that HIP_COMPILER is the same as CMAKE_CXX_COMPILER

* CMake: better this

* CI: uncomment all checks

* CMake: fix visibility of arch flags for HIP

* Doc: add subsection on HIP+CMake

* Update Docs/sphinx_documentation/source/GPU.rst

Co-authored-by: Shreyas Ananthan <shreyas@umd.edu>

* CMake: fix incorrect configuration option passed to cmake

* CI: target 'tutorials' no longer exists

Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Shreyas Ananthan <shreyas@umd.edu>
maikel added a commit to maikel/amrex that referenced this pull request Mar 5, 2021
jmsexton03 referenced this pull request in jmsexton03/amrex May 18, 2021
etpalmer63 referenced this pull request in etpalmer63/amrex Oct 13, 2021
WeiqunZhang added a commit that referenced this pull request Nov 7, 2022
It can turn "N5amrex3BoxE" into "amrex::Box", and "Z4mainEUlvE_" into
"main::{lambda()#1}".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants