Support for Gromacs's TNG format

[jbarnoud]

Gromacs is slowly moving toward the new TNG trajectory format. The format got introduced in the late releases of the 4 serie, and should become the default format eventually.

TNG allows better compression that XTC, and have random frame access built in. It also allows to store arbitrary data along side the trajectory (which may require #785), and time-dependant topology (which would require #864).

A library is provided by the gromacs people: https://github.com/gromacs/gromacs/tree/master/src/external/tng_io

See the paper: http://onlinelibrary.wiley.com/doi/10.1002/jcc.23495/full

Full support for the TNG format can be implemented in parts:

• read trajectory with constant topology
• write trajectory with constant topology
• read constant topology
• read trajectory with time-dependant topology
• write trajectory with time-dependant topology
• read arbitrary data
• write arbitrary data

[orbeckst]

And according to chatter on gmx-dev, it's being implemented in mdtraj.

[khuston]

I took a minimal crack at this with some cython that exposes the coordinates to python. (no actual MDAnalysis reader though):
https://gist.github.com/khuston/fde287d32f346a7c2dba7a6182518d46
I guess we'd include the tng_io library headers and source to be built when MDAnalysis is installed?

Looking at the MDTraj progress, it looks like their reader is nearly complete if not entirely complete.

[richardjgowers]

@khuston that looks like a good start. You could keep the tng/Cython as a small interface like @kain88-de did here:

https://github.com/MDAnalysis/mdanalysis/blob/develop/package/MDAnalysis/lib/formats/libmdaxdr.pyx

Then write a Reader which uses this interface like here:

https://github.com/MDAnalysis/mdanalysis/blob/develop/package/MDAnalysis/coordinates/XTC.py

Don't worry about how to incorporate the Reader into MDA yet, that can wait until it's working nicely.

WRT tests, you should be able to convert the XTC file in the tests to a TNG and just steal borrow all the XTC tests

[orbeckst]

Is it realistic to have basic TNG support (at least reading fixed number of atoms) in 0.17.0? It would certainly look very good on the feature list.

[hmacdope]

Can be closed with #3765

[jbarnoud]

Can be closed with #3765

Not really. #3765 only ticks the first box from the todo list.

[hmacdope]

Fair point.

[hmacdope]

Of the remaining I would say the ordering in terms of priority would go

• read topology
• write trajectory
• write topology
• everything else

I don't think time dependent topology is something we can really handle either right? So I would push that down the list a bit.
Sound about right?

[jbarnoud]

Time-dependent topologies will require a fairly major overhaul of the whole library. Let's not go there for now.

[orbeckst]

Fair enough, I unlinked this issue from PR #3765.

This issue will be an forever-open issue...