Nighly wheels generation

[galleon]

🚀 Feature

Modify/create action to generate nighlly wheels

Motivation

New releases usually happen few times a year. It might be tedious for users to build themselves a release to benefit from new features. Having nightly releases would help close that gap.

Pitch

• everyday at 1am (crontab) if master has changed a release (set of wheels) is generated
• This set of wheels is released as a zip file
• The associated documentation is generated and published on GH pages under the dev tagname
• Only the latest 7 daily wheels are kept (but only the latest documentation is available online)

Alternatives

Release more often :)

Additional context

None

[dbarbier]

Another alternative: build wheels each time master is changed, which would not require a cron job.

Why keep 7 daily wheels? A single one is enough in my opinion.

[galleon]

I believe you mean publish ... because wheels are already built each time master is modified.
Regarding the number of wheels, it seems that most projects keep several versions (stasmodels & scikit-learn for example) and build them once a day (scikit-learn).

[dbarbier]

Can you please provide links? I cannot find daily wheels for these projects.

[galleon]

You can find some here: https://anaconda.org/scipy-wheels-nightly/repo?type=pypi&label=main

I did not find any discussion on the strategy for keeping so many so I am curious. I don't find this very responsible either. I like the idea of storing them on GH because we can delete them which is not possible on PyPi (to be checked for conda but that would mean an other infrastructure)

I therefore propose to continue on GH and decide on the strategy:

• all between to PyPi releases triggered by master changes + cron
• less : 7, 1 ?

[dbarbier]

These wheels are not hosted on GH, why mimic these projects? Unless there is a valid reason for keeping older wheels, the simplest solution is to only keep the latest one.

[galleon]

Did I say that we should mimic those projects? I am just wondering why those projects are keeping such a long history ... these are not small projects!
At least it seems obvious that if we release some nightly wheels some users will probably try them and we should keep them until the next official release is out in case some problems are highlighted (as I want to test on the same wheel) - I cannot expect nor request that they should use the latest nightly build.

[dbarbier]

Did I say that we should mimic those projects?

Yes. When I asked why you want to keep several wheels, you replied that other projects do so.

[...]

At least it seems obvious that if we release some nightly wheels some users will probably try them and we should keep them until the next official release is out in case some problems are highlighted (as I want to test on the same wheel) - I cannot expect nor request that they should use the latest nightly build.

That does not make sense to me, it is in contradiction with your proposal to keep only 7 wheels.

[galleon]

I would like to close this one. I need from @fteicht the strategy he would like to implement. I see two options between releases:

1. generate a new nightly wheels each time a PR is merged in master and keep them all - they are then all deleted when a new release is created
2. generate a new nightly wheels each time a PR is merged in master and keep only the latest.

Both can be implemented. Option 2 is simpler from an end-user stand point since you don't have to choose a file (you will automatically pick the latest one) a downside is that if it is buggy you cannot get back to the previous one,

Option 1 is a bit more difficult for the user in case of multiple nightlys between 2 releases.

Proposed name for the wheels is nightly_YYYY-MM-DD_distance_hash.zip where:

• YYY-MM-DD is the date at which the wheel is generated
• distance is the number of commits since lastest release
• hash is the hash from latest commit

Option 1 or option 2 That is the question ?

[dbarbier]

Nightly wheels have been implemented, this issue can be closed.

[galleon]

Solved by #95 and #144