Updates to the setup procedure for building and installing the package (

#4204)

Overview of the changes in this commit:

 * Limit `find_packages` in `setup.py` to only the `aiida` package
 * Unvendor the `fastentrypoints` dependency
 * Remove files from the `MANIFEST.in`

The `setup.py` uses the `setuptools.find_packages` function to determine
all packages that need to be included automatically. Since we don't
specify any rules, it finds all packages in the current directory. This
means that `docs`, `tests`, and `utils` are currently also installed. To
avoid this, we use the `include` keyword to only find `aiida` package
and all of its sub-packages.

We have been using the `fastentrypoints` dependency as a hack to ensure
that console scripts that are installed through entry points, such as
`verdi` in our case, have a fast start up time. This is critical for
tab-completion. The reason this hack is necessary is because setuptools,
which creates the wrapper script, uses `pkg_resources` by default, which
is really slow and adds 200 ms of overhead. The `fastentrypoints` monkey
patches `setuptools` to generate a faster wrapper script. Note that none
of this applies if the package is installed through a wheel.

For this reason we have been vendoring `fastentrypoints` but we now move
it to a build requirement in `pyproject.toml`, which ensures that it can
be imported whenever the package is installed through `pip`. This incurs
a slight overhead of having to install `fastentrypoints` even when the
package is installed as a wheel, but we no longer have to vendor it.

Finally, we remove the following files from the distribution by removing
them from the manifest:

 * The `*.aiida` export files which had already been moved to `tests/`
 * The `script.py.mako` which is only needed for devs to create migration
   files for the SqlAlchemy backend
 * The `*.png` files from the `docs` directory
 * The files in `utils` which are only necessary for developers

In addition to removing the images from the `docs/` we change the readme
to use a permalink for the AiiDA logo from the website instead of the
relative path to the docs, such that the logo also displays correctly
on, among others, PyPI.

MANIFEST.in

@@ -1,9 +1,5 @@
-include aiida/backends/tests/export_import_test_files/*.aiida
-include aiida/backends/sqlalchemy/migrations/script.py.mako
 include aiida/cmdline/templates/*.tpl
 include aiida/manage/backup/backup_info.json.tmpl
-include docs/source/images/*.png
-include utils/*
 include setup.json
 include AUTHORS.txt
 include CHANGELOG.md

README.md

@@ -1,4 +1,4 @@
-# <img src="docs/source/images/logo_aiida_main.png" alt="AiiDA" width="200"/>
+# <img src="http://www.aiida.net/wp-content/uploads/2020/06/logo_aiida.png" alt="AiiDA" width="200"/>
 
 AiiDA (www.aiida.net) is a workflow manager for computational science with a strong focus on provenance, performance and extensibility.
 

open_source_licenses.txt

@@ -9,9 +9,6 @@ Django:
 Python:
   * aiida/transports/transport.py
 
-Fastep:
-  * utils/fastentrypoints.py
-
 The respective copyright notices are reproduced below.
 
 --------------------------------------------------------------------------
@@ -328,33 +325,3 @@ FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
 OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
-
--------------------------------------------------------------------------------
-
-fastentrypoints.py Licence:
-
- Copyright (c) 2016, Aaron Christianson
- All rights reserved.
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions are
- met:
-
- 1. Redistributions of source code must retain the above copyright
-    notice, this list of conditions and the following disclaimer.
-
- 2. Redistributions in binary form must reproduce the above copyright
-    notice, this list of conditions and the following disclaimer in the
-    documentation and/or other materials provided with the distribution.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
- IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
- TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
- PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
- TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
- PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
- LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
- NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
- SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

pyproject.toml

@@ -1,3 +1,3 @@
 [build-system]
-requires = [ "setuptools>=40.8.0", "wheel", "reentry~=1.3",]
+requires = [ "setuptools>=40.8.0", "wheel", "reentry~=1.3", "fastentrypoints~=0.12",]
 build-backend = "setuptools.build_meta:__legacy__"

setup.py

@@ -13,7 +13,12 @@
 import os
 import sys
 
-from utils import fastentrypoints  # pylint: disable=unused-import
+try:
+    import fastentrypoints  # pylint: disable=unused-import
+except ImportError:
+    # This should only occur when building the package, i.e. when
+    # executing 'python setup.py sdist' or 'python setup.py bdist_wheel'
+    pass
 from setuptools import setup, find_packages
 
 if (sys.version_info.major, sys.version_info.minor) == (3, 5):
@@ -45,7 +50,7 @@
     EXTRAS_REQUIRE['all'] = list({item for sublist in EXTRAS_REQUIRE.values() for item in sublist if item != 'bpython'})
 
     setup(
-        packages=find_packages(),
+        packages=find_packages(include=['aiida', 'aiida.*']),
         long_description=open(os.path.join(THIS_FOLDER, 'README.md')).read(),
         long_description_content_type='text/markdown',
         **SETUP_JSON

utils/dependency_management.py

@@ -177,7 +177,8 @@ def generate_pyproject_toml():
 
     pyproject = {
         'build-system': {
-            'requires': ['setuptools>=40.8.0', 'wheel', str(reentry_requirement)],
+            'requires': ['setuptools>=40.8.0', 'wheel',
+                         str(reentry_requirement), 'fastentrypoints~=0.12'],
             'build-backend': 'setuptools.build_meta:__legacy__',
         }
     }