PyPi releases do not always match repo code

[mal]

While trying to work out why installing 0.0.17 still triggered the issue outlined in #302, I started looking at other versions and discovered that a few of them don't appear to match the code tagged for that release in this repo.

I've done a rudimentary check on the last 5 versions by doing a very lazy test, just checking if the version of pymysql matches between the git tag and the pypi release, here are the results:

0.0.15: match

$ docker container run -it --rm python pip install aiomysql==0.0.15 | grep -F 'Collecting PyMySQL'                                                                                                                 
Collecting PyMySQL>=0.7.5 (from aiomysql==0.0.15)

0.0.16: doesn't match

$ docker container run -it --rm python pip install aiomysql==0.0.16 | grep -F 'Collecting PyMySQL'
Collecting PyMySQL<0.9,>=0.7.5 (from aiomysql==0.0.16)

0.0.17: doesn't match

$ docker container run -it --rm python pip install aiomysql==0.0.17 | grep -F 'Collecting PyMySQL'
Collecting PyMySQL<=0.9.2,>=0.9 (from aiomysql==0.0.17)

0.0.18: no release

$ docker container run -it --rm python pip install aiomysql==0.0.18 | grep -F 'Collecting PyMySQL'
err: No matching distribution found for aiomysql==0.0.18

0.0.19: match

$ docker container run -it --rm python pip install aiomysql==0.0.19 | grep -F 'Collecting PyMySQL'
Collecting PyMySQL<=0.9.2,>=0.9 (from aiomysql==0.0.19)

At best this is awfully confusing because according to the code 0.0.17 was the first release to have the conservative version pinning for pymysql, but in fact that's some how made it into the 0.0.16 release (which the code doesn't reflect) but not into 0.0.17.

It's a little unsettling to find that the versions released in PyPi don't always match the tagged code releases. Is this some sort of mistake or quirk in the shipping process? 😕

Possibly related: #318

[terricain]

0.0.17 was my bad, I attempted to build 0.0.18 but forgot to versionbump.

[terricain]

@jettify is it worth getting travis to somehow republish 0.0.17?

[webknjaz]

@terrycain you cannot edit published dists.

[jettify]

At first, I did deploy manually so something could be missing, but now we have build script so, I believe the situation will improve, also @terrycain watching my back now.

[jettify]

We still need to manually bump version and updated changes, if someone has ideas how to improve this please share.

[webknjaz]

@jettify I have a completely tested and working flow where the only thing needed is just pushing a new tag. It's a single source of truth.

[terricain]

We still need to manually bump version

I've solved that in the past with bumpversion which has always worked for me :D

[webknjaz]

Which proves to be error-prone because of human presence in deployment chain.

[webknjaz]

@jettify @terrycain do you still want hardcoded version based flow?

[terricain]

@webknjaz im not too fussed, if we can push a tag and thats all, as you said less error prone is better.

You got an example of that magical solution, im interested.

[webknjaz]

I've been using this flow with setuptools-scm plugin for years. Andrew doesn't want it for aiohttp, but I'm pretty sure that other repos can rely on this all long as their maintainers want this. I can prepare a PR if there's no objections.

[terricain]

Sure, as long as @jettify has no objections

[webknjaz]

@jettify?

[webknjaz]

It looks like @jettify doesn't care...

[terricain]

Sure raise pr :).

I converted one of my projects to setuptools SCM the other day. Really nice

[jettify]

I am on long business trip, do not have lots of time right now. Will take a look once back to the town.

…

On Nov 19, 2018, at 11:00, Sviatoslav Sydorenko ***@***.***> wrote: It looks like @jettify doesn't care... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[webknjaz]

@jettify still there?

[webknjaz]

@jettify?

[jettify]

So setuptools-scm capable to fetch git tag and bump version in init file?

[webknjaz]

with setuptools-scm you don't need to store version in a python file (but there's an option to do so). you can grab the installed version via pkg_resources.

[jettify]

I think having version in __init__ file is important other then that, how that version is bumped I do not mind.

[webknjaz]

so when current commit is tagged, setuptools sets version to that tag value, if it's not tagged then version will be tag+patch_bump+local_version_spec.
so for v1.0.0 it'd be 1.0.0, but for the next commit it'd be 1.0.1dev0+ghash.date

[webknjaz]

So I'll do smth like https://github.com/pytest-dev/pytest/blob/master/setup.py#L29 then

[jettify]

But version is not present in git ...

[webknjaz]

And yet it's in sync with the source

[jettify]

Is it possible to have commit that bumps version?

[webknjaz]

Why would you need it? It creates desynchronized source

[webknjaz]

You need a version in dist because it doesn't have Git tree context. Git itself has tags. It'd be pointless to have a file different from a tag.

[asvetlov]

@webknjaz does git plugin replace __version__ in top-level __init__.py with an actual release number when sdist tarball is uploaded to pypi?

[webknjaz]

Almost: it generates a new module. So I'd not combine it with other code and have a separate file for version.

[webknjaz]

e.g. __version__.py or so

[webknjaz]

@jettify @asvetlov so?

[asvetlov]

In my experience release publishing requires a little more work than just version pinning.
At least changelog should be updated.
It needs extra manual steps. Changing a version string in __init__.py is not a big deal taking into account all other actions.
It just not my point of pain honestly.

[webknjaz]

You should always keep in mind to match the tag version with source anyway. I think that if you maintain a ton of dists this simple thing makes release process much easier.
As for changelog, I think this should be automated as well. I'm already working on a bot which enforces change fragments presence in PRs, at some point I hope to have an App or an Action improving the release as well.

[webknjaz]

@jettify ?

[asvetlov]

I love automation but version pinning takes less than 30 seconds for every release.
aiomysql has 19 releases so far, ~10 mins.
We all spent more time discussing it. The similar for changelog update (which needs additional hours for a tool writing).