Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish Python 3 compatible packages and migrate to CircleCI 2.0 #41

Merged
merged 14 commits into from
May 22, 2018
Merged
151 changes: 151 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# CircleCI 2.0 configuration file. See <https://circleci.com/docs/2.0/language-python/>.
version: 2
jobs:
build_python2:
docker:
- image: python:2.7.14

steps:
- checkout

- restore_cache:
key: v1-py2-{{ checksum "setup.py" }}

- run:
name: Create a virtualenv
command: |
mkdir -p /tmp/venv/country_template
virtualenv /tmp/venv/country_template
echo "source /tmp/venv/country_template/bin/activate" >> $BASH_ENV

- run:
name: Install dependencies
command: |
pip install --upgrade pip twine wheel
pip install --editable .[test] --upgrade
# pip install --editable git+https://github.com/openfisca/openfisca-core.git@BRANCH_NAME#egg=OpenFisca-Core # use a specific branch of OpenFisca-Core

- save_cache:
key: v1-py2-{{ checksum "setup.py" }}
paths:
- /tmp/venv/country_template

- run:
name: Run tests
command: make test

- run:
name: Check version number has been properly updated
command: |
git fetch
.circleci/is-version-number-acceptable.sh

deploy_python2:
docker:
- image: python:2.7.14
environment:
PYPI_USERNAME: openfisca-bot # Edit this value to replace it by your Pypi username
# PYPI_PASSWORD: this value is set in CircleCI's web interface; do not set it here, it is a secret!

steps:
- checkout

- restore_cache:
key: v1-py2-{{ checksum "setup.py" }}

- run:
name: Check for functional changes
command: if .circleci/detect-functional-changes.sh ; then circleci step halt ; fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we just .circleci/detect-functional-changes.sh? Shouldn't deployment fail?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright I understood. No, it should not fail, it should be green but not do anything. Gotcha.


- run:
name: Upload a Python package to Pypi
command: |
source /tmp/venv/country_template/bin/activate
.circleci/publish-python-package.sh

- run:
name: Publish a git tag
command: .circleci/publish-git-tag.sh

build_python3:
docker:
- image: python:3.6

steps:
- checkout

- restore_cache:
key: v1-py3-{{ checksum "setup.py" }}

- run:
name: Create a virtualenv
command: |
mkdir -p /tmp/venv/country_template
python -m venv /tmp/venv/country_template
echo "source /tmp/venv/country_template/bin/activate" >> $BASH_ENV

- run:
name: Install dependencies
command: |
pip install --upgrade pip twine wheel
pip install --editable .[test] --upgrade
# pip install --editable git+https://github.com/openfisca/country-template.git@BRANCH_NAME#egg=OpenFisca-Country-Template # use a specific branch of OpenFisca-Country-Template

- save_cache:
key: v1-py3-{{ checksum "setup.py" }}
paths:
- /tmp/venv/country_template

- run:
name: Run tests
command: make test

- run:
name: Check version number has been properly updated
command: |
git fetch
.circleci/is-version-number-acceptable.sh

deploy_python3:
docker:
- image: python:3.6
environment:
PYPI_USERNAME: openfisca-bot # Edit this value to replace it by your Pypi username
# PYPI_PASSWORD: this value is set in CircleCI's web interface; do not set it here, it is a secret!

steps:
- checkout

- restore_cache:
key: v1-py3-{{ checksum "setup.py" }}

- run:
name: Check for functional changes
command: if .circleci/detect-functional-changes.sh ; then circleci step halt ; fi

- run:
name: Upload a Python package to Pypi
command: |
source /tmp/venv/country_template/bin/activate
.circleci/publish-python-package.sh

workflows:
version: 2
build_and_deploy:
jobs:
- build_python2
- build_python3
- deploy_python2:
requires:
- build_python2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to publish a Python2 package that does not pass Python3 builds. How would we manage a 23.1.2 tag that got published in Python2 but failed in Python3? We would then have to publish a 23.1.3 tag that would be available in both environments… So all version numbers could co-exist differently in each language version 😕

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I misinterpreted a previous comment.

- build_python3
filters:
branches:
only: master
- deploy_python3:
requires:
- build_python2
- build_python3
filters:
branches:
only: master
10 changes: 10 additions & 0 deletions .circleci/detect-functional-changes.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#! /usr/bin/env bash

VERSION_CHANGE_TRIGGERS="setup.py MANIFEST.in openfisca_country_template"

last_tagged_commit=`git describe --tags --abbrev=0 --first-parent` # --first-parent ensures we don't follow tags not published in master through an unlikely intermediary merge commit

if git diff-index --shortstat $last_tagged_commit -- $VERSION_CHANGE_TRIGGERS ":(exclude)*.md"
then echo "No functional changes detected."
else exit 1
fi
Original file line number Diff line number Diff line change
@@ -1,16 +1,22 @@
#! /usr/bin/env bash

VERSION_CHANGE_TRIGGERS="setup.py MANIFEST.in openfisca_country_template"
if [[ $CIRCLE_BRANCH == master ]]
then
echo "No need for a version check on master."
exit 0
fi

if git diff-index --quiet origin/master -- $VERSION_CHANGE_TRIGGERS ":(exclude)*.md"
then exit 0 # there are no changes at all, the version is correct
if $(dirname "$BASH_SOURCE")/detect-functional-changes.sh
then
echo "No need for a version update."
exit 0
fi

current_version=`python setup.py --version`

if git rev-parse --verify --quiet $current_version
then
echo "Version $current_version already exists:"
echo "Version $current_version already exists in commit:"
git --no-pager log -1 $current_version
echo
echo "Update the version number in setup.py before merging this branch into master."
Expand All @@ -20,7 +26,7 @@ fi

if git diff-index --quiet origin/master CHANGELOG.md
then
echo "CHANGELOG.md has not been modified, while the code has changed."
echo "CHANGELOG.md has not been modified, while functional changes were made."
echo "Explain what you changed before merging this branch into master."
echo "Look at the CONTRIBUTING.md file to learn how to write the changelog."
exit 2
Expand Down
4 changes: 4 additions & 0 deletions .circleci/publish-git-tag.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#! /usr/bin/env bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you're not using bash-specific instructions, sh is safer: bash is an implementation with extensions while sh is an interface.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add that usually sh is just a pointer to bash, but not always.


git tag `python setup.py --version`
git push --tags # update the repository version
4 changes: 4 additions & 0 deletions .circleci/publish-python-package.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
#! /usr/bin/env bash

python setup.py bdist_wheel # build this package in the dist directory
twine upload dist/* --username $PYPI_USERNAME --password $PYPI_PASSWORD # publish
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Changelog

### 3.1.0 - [#41](https://github.com/openfisca/country-template/pull/41)

* Make package compatible with Python 3

### 3.0.2 - [#37](https://github.com/openfisca/country-template/pull/37)

* Declare package compatible with OpenFisca Core v23
Expand Down
26 changes: 0 additions & 26 deletions circle.yml

This file was deleted.

11 changes: 0 additions & 11 deletions deploy-if-version-bump.sh

This file was deleted.

4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

setup(
name='OpenFisca-Country-Template',
version='3.0.2',
version='3.1.0',
author='OpenFisca Team',
author_email='contact@openfisca.fr',
description=u'OpenFisca tax and benefit system for Country-Template',
Expand All @@ -16,7 +16,7 @@
url='https://github.com/openfisca/openfisca-country-template',
include_package_data = True, # Will read MANIFEST.in
install_requires=[
'OpenFisca-Core >= 23, < 24.0',
'OpenFisca-Core >= 23.1, < 24.0',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for that change here or is it just a knight? 😛

Copy link
Member Author

@fpagnoux fpagnoux May 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changelog claims that we are making this package compatible with python 3.
We thus need to make sure we request a core version that is compatible with python 3.

],
extras_require = {
'api': [
Expand Down