Release Magpie 2.0

[jstypka]

As mentioned before (in #97), Keras 2.0 that Magpie depends on introduced some breaking changes, so in order to keep up to date with it, we need to release a new version of Magpie.

There are no changes to functionality, only updates to the API so it works with new Keras. I also took advantage to change the main object name from MagpieModel to Magpie which I wanted to do for a long time. These are breaking changes, therefore we release them under v2.0. Version 1.0 is tagged and can still be installed from GHE.

Changes:

• MagpieModel -> Magpie as a name of the main object
• nb_epochs -> epochs as in the new Keras
• rename a few keyword arguments in the Keras layers
• rewrite the NN model definitions to conform to the new Keras functional API
• update the batch_train() function, which now takes in the number of batches and not the number of samples
• add a paragraph in the docs explaining the difference between v1.0 and v2.0
• added a test for batch_train() that uses an RNN
• bump the version in setup.py to 2.0 🍾

@kaplun

[jstypka]

so how are things looking @kaplun ? :)

[kaplun]

Ahem! I forgot about it, and today was fully away at a course. And tomorrow will also away... d'oh!

[jstypka]

no rush, take a look when you have some free time :-)

[kaplun]

LGTM! (I trust you for the NN part!)

[kaplun]

@inspirehep/core anyone with experience in NN wants to have a second look?

[NowayIndustries]

So purely from a user's perspective: All I have to do to start using the shiny new Magpie 2.0 is:

• Modify all calls to train, replacing nb_epochs with epochs
• Modify all calls to batch_train, replacing nb_epochs with epochs
• Replace all imports for MagpieModel with Magpie

and thats it? No changes to Magpie's output (like when Magpie switched to top-k accuracy, the indexes in the Keras callback changed) by switching to a new Keras version?

---

Perhaps this information could be added to the README for me developers using Magpie? Otherwise I developers would have to go digging around in closed pull requests for information.

I think a simple summation of changes that have to be made should suffice, adding a new chapter below the chapter about differences between v1 and v2. Perhaps something like:

Migrating from Magpie v1.0 to v2.0

• Modify all calls to train, replacing nb_epochs with epochs
• Modify all calls to batch_train, replacing nb_epochs with epochs
• Replace all import for MagpieModel with Magpie

[jstypka]

@NowayIndustries Yes, that's pretty much it. I intend to create a GitHub release and describe all the changes there, but perhaps adding another paragraph to the README is also a sensible idea.

[jstypka]

@NowayIndustries if you intend to migrate, let me know how the new version works for you!

[NowayIndustries]

@jstypka Unless anything else pops up I'll probably be working on migrating to the new version next Thursday/Friday. Hopefully it wll indeed be as simple as those small changes, I'll let you know either way.

[NowayIndustries]

So as it turns out it is that simple, just follow the 3 simple steps and you're done.

One small caveat though, you can't load saved v1 models into v2 (not that you'd really want to in any normal case, but my API around Magpie autoloads the files if found).
The terminal will be spammed (for every incorrect layer name) as per Keras2 release notes, and a ValueError('You are trying to load a weight file containing 6 layers into a model with 1 layers.',) will be thrown.

I'm currently running a copy of production with v2 as production (with the models retrained obviously) and so far so good. Training & prediction both work fine and predicted labels seem relevant and at least similar (if not identical) to the v1 variant (of course including some wiggle room for differences between training sessions).

So 🎉 congratulations 🎉, it just works (TM).

For completeness sake:
python --version

Python 3.5.2

pip freeze

bleach==1.5.0
boto==2.48.0
bz2file==0.98
certifi==2017.7.27.1
chardet==3.0.4
click==6.7
clickclick==1.2.2
connexion==1.0.129
decorator==4.1.2
Flask==0.12.2
Flask-HTTPAuth==3.2.2
gensim==0.13.4.1
h5py==2.7.1
html5lib==0.9999999
idna==2.6
ipython-genutils==0.2.0
itsdangerous==0.24
Jinja2==2.9.6
jsonschema==2.6.0
jupyter-core==4.3.0
Keras==2.0.8
magpie==2.0
Markdown==2.6.9
MarkupSafe==1.0
nbformat==4.4.0
nltk==3.2.5
numpy==1.13.3
pkg-resources==0.0.0
plotly==2.1.0
protobuf==3.4.0
python-dateutil==2.6.0
pytz==2017.2
PyYAML==3.12
requests==2.18.4
scikit-learn==0.19.0
scipy==0.19.1
six==1.11.0
smart-open==1.5.3
strict-rfc3339==0.7
swagger-spec-validator==2.1.0
tensorflow==1.3.0
tensorflow-tensorboard==0.1.8
traitlets==4.3.2
urllib3==1.22
Werkzeug==0.12.2

I had to modify a grand total of 12 lines in order to apply these changes, of which 7 were documentation changes ;)

[jstypka]

Awesome @NowayIndustries, thanks for the heads up! Let us know if you also see any discrepancies in terms of accuracy, both drop and an increase - that would be interesting.

@kaplun I don't know what state is Magpie at the model in production at CERN, but the incompatibility between the models is worth noting.