Bugfix: monitor the correct sub-domain with ProbeVolume
#144
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I found a minor bug in the `ProbeVolume` class. Before this pull-request, the probe was monitoring a field solution in a sub-domain different than the one configured in the YAML input file. The bug only affects simulations with more than one MPI process, when we want to monitor the solution in a sub-volume of the computational domain. The bug was introduced in version `0.4` of PetIBM.
|
Nice one! 🚀 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I found a minor bug in the
ProbeVolumeclass.Before this pull-request, the probe was monitoring a field solution in a sub-domain different than the one configured in the YAML input file.
The bug only affects simulations with more than one MPI process, when we want to monitor the solution in a sub-volume of the computational domain.
The bug was introduced in version
0.4of PetIBM.PetIBM implements a class called
ProbeVolumethat can be used to monitor more frequently the field solution in a sub-region of the computational domain.This is done by creating a sub-vector of a given PETSc Vec object with the routine
VecGetSubVector.The routine takes an index set (
IS) as input.Before this pull-request, the index set stored the index of the points located in the sub-domain using the natural ordering.
However, we need to provide the indices following the PETSc ordering, not the natural ordering.
(Natural ordering and PETSc ordering are different when using more than one MPI process.)
To convert the list of integer indices from natural ordering to PETSc ordering, we now (1) create an Application Ordering object from the DMDA on which is based the field to monitor and (2) use the routine
AOApplicationToPetsc.To illustrate the problem, I used the 2D flow around a cylinder (Re=40) as an example.
I ran the
petibm-decoupledibpmexecutable to compute 1000 time steps, with PetIBM-0.4 and the version provided in this patch, using 4 MPI processes.During the simulation, the x-component of the velocity vector field was monitored in the sub-domain [0.691297, 0.782822] x [-1.033947, 1.033947].
The solution in the sub-domain was accumulated over 100 time steps, averaged, and then saved to a HDF5 file.
The figures below were obtained with PetIBM-0.4 (top) and this commit (bottom).
The left panel shows the contour of the x-velocity after 1000 time steps.
On top of this field, we plot the contour of the monitored solution in the sub-domain (which corresponds to the time-averaged solution between 900 and 1000 time steps).
Using PetIBM-0.4, the sub-domain solution does not match the full-domain solution; it does with this commit.
The right panel of the figures show the profile of the x-velocity behind the cylinder (at x/c = 0.75) after 900 and 1000 time steps, as well as the profile from the probe.
With this commit, the velocity profile from the probe is now similar to the ones after 900 and 1000 time steps.