Hybrid Parallel AD (Part 1/?) #1214
Conversation
|
The commits so far address parts of OpDiLib's incorporation into the AD workflow and sort out some additional issues along the lines of the |
|
It looks like the builds are failing because of swig (python wrapper) and a missing definition of size_t. |
|
True that, I built it without the python wrapper locally. This build command worked for me. The "T" in OMPT stands for "tools", see Chapter 4 of the OpenMP 5.1 specification. We also explain it with a focus on AD in our preprint. |
|
I see, then it should be possible to detect the version of the standard supported by the compiler and only enable the feature in that case. We do that for simd directives for compatibility (hopefully) with the MS compilers. |
| /*--- Note, passiveDoubleBuffer and doubleBuffer point to the same address. | ||
| * This is the reason why we have to do the following copy/reordering in two steps. ---*/ | ||
| /*--- Reorder the data in the buffer. ---*/ | ||
|
|
||
| /*--- Step 1: Extract the underlying double value --- */ | ||
| vector<passivedouble> tmpBuffer(nPoint_Recv[size]); | ||
|
|
||
| if (!std::is_same<su2double, passivedouble>::value){ | ||
| for (int jj = 0; jj < VARS_PER_POINT*nPoint_Recv[size]; jj++){ | ||
| const passivedouble tmpVal = SU2_TYPE::GetValue(doubleBuffer[jj]); | ||
| passiveDoubleBuffer[jj] = tmpVal; | ||
| /*--- For some AD datatypes a call of the destructor is | ||
| * necessary to properly delete the AD type ---*/ | ||
| doubleBuffer[jj].~su2double(); | ||
| } | ||
| } | ||
|
|
||
| /*--- Step 2: Reorder the data in the buffer --- */ |
There was a problem hiding this comment.
These last changes were to fix issues when the RealReverseIndex is used.
We used to have a char buffer that was used for both passive and active doubles, which required these explicit calls to the destructor, but I guess that did not work so well with mpi and all.
Now there is only passivedouble, which means that once something goes into the data sorter AD information is lost.
This would matter if some computation was performed with time averaged values, which does not seem to be the case at the moment...
| void SetUnsorted_Data(unsigned long iPoint, unsigned short iField, su2double data){ | ||
| connSend[Index[iPoint] + iField] = data; | ||
| connSend[Index[iPoint] + iField] = SU2_TYPE::GetValue(data); | ||
| } | ||
|
|
||
| su2double GetUnsorted_Data(unsigned long iPoint, unsigned short iField) const { | ||
| passivedouble GetUnsorted_Data(unsigned long iPoint, unsigned short iField) const { | ||
| return connSend[Index[iPoint] + iField]; | ||
| } |
There was a problem hiding this comment.
This is were the active-passive conversion takes place.
| CFEMDataSorter* volumeSorter; //!< Pointer to the volume sorter instance | ||
| const CFEMDataSorter* volumeSorter; //!< Pointer to the volume sorter instance |
|
This pull request fixes 8 alerts when merging 45cc9a5 into e823be7 - view on LGTM.com fixed alerts:
|
| if (solver[iZone][iInst][MESH_0][ADJFEA_SOL]) { | ||
| solver[iZone][iInst][MESH_0][ADJFEA_SOL]->SetRecording(geometry[iZone][iInst][MESH_0], config[iZone]); | ||
| } |
There was a problem hiding this comment.
I already thought I dreamed of removing this code bit when I scrolled through #1257 and it was gone (because it was already deleted here) ... that conditional evaluated to true btw and it does not crash for non-FEA cases . Just c++ things
Proposed Changes
The goal of this PR is to establish AD support for the OpenMP features of SU2.
AD support in SU2 is provided by CoDiPack, which is coupled with MeDiPack for the differentiation of MPI. OpenMP support is added by coupling it in addition with OpDiLib so that all in all, hybrid parallel AD is achieved.
This involves at least the following steps.
Related Work
OpenMP features introduced in #789, #824, #830, #834, #843, #861, #895, #975, #1178, possibly others
PR Checklist