ADIOS2: Write/Put In Sync Mode

[guj]

Using the Sync mode for now does:

• save an additional memory copy (application memory footprint)?
• has no performance impact currently (as of ADIOS 2.6.0)?
• is more robust than the default deferred mode?

[ax3l]

Please see a suggestion inline. For such tweaks, we need to document a bit more background info (when it occurred and what)

[ax3l]

This might be based on a misunderstanding, we are currently clarifying.

[guj]

OK, since deferred mode will stay in ADIOS and openPMD is not going to run into data corruption issues due to misusing this mode, I will close this pull request. @ax3l

[ax3l]

Sounds good, thanks!
openPMD::Series::flush(); is the API contract in openPMD-api that is equivalent to PerformPuts() in ADIOS2: it gives the shared users buffers free again.

The name is not ideal in our API, which is based on a misunderstanding at some point in the past. Our API (as well as ADIOS') does not make a guarantee when things will arrive on disk (the historical meaning of "flush"). It should also not make a guarantee that written data is at this point "consistent to openPMD" - but it sometimes does this as of now.

The idea is to trigger completion of the latter with additional ::close() methods #778 #746, e.g. by writing additional openPMD information that was not changed from their defaults.

We are planning a simple always-sync mode #484 that just flush()es the provided data buffers synchronously.
openPMD conformance would be at latest be ensured at Series, Iteration or Record closure, in the order: (implicit) close and (implicit) flush.