Convert package release repositories from Mercurial to git and make public

[fingolfin]

Apparently, the package distribution is currently handled by a single mercurial repository which aggregates all packags. This repository seems to be private (e.g. I don't have access to it, in the sense that I do not even know where it is located).

This means that in particular virtually nobody can test the scripts in here.

Hence, I think we should make this repository public. And while at it, let's convert it from hg to git so that we us git everywhere (one less skill needed to work on the package distribution).

I am willing to help with both converting the package repository, and adapting the scripts to use git instead of hg; but alas, as I already said, I don't even know where that repository is, and cannot test the existing (mostly undocumented, it seems?) scripts without that.

[olexandr-konovalov]

Apparently, the package distribution is currently handled by a single mercurial repository which aggregates all packags. This repository seems to be private (e.g. I don't have access to it, in the sense that I do not even know where it is located).

No, there are individual repositories for each package. Yes, they are private mercurial repos. They were on yin and since yesterday moved to our new Jenkins slave which is the new location where checks for package updates and wrapping release candidates is taking place.

This means that in particular virtually nobody can test the scripts in here.

There are no scripts there - only package release snapshots. All scripts are in https://github.com/gap-system/gap-distribution/tree/master/DistributionUpdate/PackageUpdate.

Hence, I think we should make this repository public. And while at it, let's convert it from hg to git so that we us git everywhere (one less skill needed to work on the package distribution).

No objections to make these repos public anyway - they contain only public releases of packages and nothing else. The only thing that should be clear if anyone discovers such repository that they are not development repos and no pull requests, issue reports should go there. Maybe we need to have them on GitLab elsewhere, not on GitHub.

I am willing to help with both converting the package repository, and adapting the scripts to use git instead of hg; but alas, as I already said, I don't even know where that repository is, and cannot test the existing (mostly undocumented, it seems?) scripts without that.

Thanks. I have some other things before March 1st, so maybe in March we could work together on this.

[fingolfin]

There are no scripts there - only package release snapshots. All scripts are in https://github.com/gap-system/gap-distribution/tree/master/DistributionUpdate/PackageUpdate.

Huh? I didn't write "there", I wrote "here", i.e. inside this repository.

As to this being separate packages: OK, I remember now, you told me this at least twice before -- and I keep forgetting it, partially because AFAIK it is not documented anywhere... I find this whole thing super frustrating, because it is completely intransparent to me.

So, I think we should not bother to convert those individual package repositories, because I think they are the wrong approach to the problem. In particular, they do not capture an important aspect of the package distribution: Which precise set of package versions was tested together.

Instead, I think there should be a single repository which combines the packages together. New package updates would result in branches which merge the new package, and which could be tested; if succesful, the branch could be merged. This could even be mapped onto GitHub pull requests.

I'lll try to write up a more detailed proposal, and we could implement it during the SageDays.

[olexandr-konovalov]

Yes, this would be a proper fit into the agenda of Sage/GAP Days.

The set of repositories actually captures which precise set of package versions belongs together - each of the repos has a bookmark with a release timestamp. So I can select a collection of packages belonging to any particular release since 2012, and I can also answer for each version in which releases it was included.

[olexandr-konovalov]

@fingolfin I've added now README.md files to all directories of https://github.com/gap-system/gap-distribution/ repo - see cb733cf. Some bits were documented in the doc/dev manual, but of course that was quite obscure and incomplete.

[olexandr-konovalov]

A repository for that has been set up: https://github.com/gap-system/pkg-dist. @fingolfin has some design notes elsewhere - perhaps they will be eventually posted there.