opam root compatibilities

Discussion around #4636

Context

On versions and fields: for a long moment we had opam-version related to the opam version (opam-version: "2.0" package files needed at least opam version 2.0). Since 2.1.0~alpha, opam-version was introduced in all opam files and took the meaning of format version, the syntax of the opam file (package, repo, config, all). But for root config file, it kept the double meaning of format version and opam root version.

On 2.1 there was new fields (depext) in the config root file, so the format version bumped, and opam root version (#4266/#4280). It resulted in a root format upgrade operation. To ease this operation, a on-the-fly mechanism was introduced in #4314: for soft upgrade do them without user confirmation, and don't take a write lock on opam root (on-the-fly upgrade).

What we want

These mechanisms ease the use of a new binary with an old opam root. In the other way, opam directly fails. So with this new field we want to ease the reading of an opam root newer than the binary or library.

What is the status on versions

There is several version that we need to consider:

• opam version: the version of the binary or lib
• file format version: the low level syntax, from opam-file-format parser
• format version: the high level syntax, from opam-format parsers
• opam root version: the version of the opam root structure (files syntax and directory structure)
• readable version: last readable version for a file, ie since this version there was only field addition that are not understood/needed by an older opam version

Constraint of these versions

• file format version is related to opam-file-format release versioning
• format version bump for each changes in the file
• opam root version bump for each update in the opam root structure

opam-file-format and opam-version field

• opam-version embed a readable version meaning according opam-file-format version
• opam-file-format major version is correlated to opam major version
• opam-file-format.x & opam-version: "y" when y<=x`: normal parsing
• opam-file-format.x & opam-version: "y" when y>x`: best effort parsing
• @dra: the only thing opam.x should do once it sees opam-version: "y" where y > x is stop parsing. opam.x can only successfully parse opam-version: "y" files if it's built with opam-file-format >= y

The mechanism

• opam root version < opam version: FormatUpgrade, as usual
• opam root version = opam version: nothing
• opam root version > opam version: If write lock required error message. Otherwise (none or read lock)
  • readable version <= opam version: read without error, and do the operation
  • readable version > opam version: error

Fields to set up