Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/add option: do not compile fim c-extension #494

Merged
merged 11 commits into from
Oct 13, 2022
1 change: 0 additions & 1 deletion .github/workflows/build_wheels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ on:
# Building wheels on Ubuntu and Windows systems
jobs:
build_wheels:
if: github.event.review.state == 'approved'
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
Expand Down
33 changes: 31 additions & 2 deletions doc/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ Elephant requires `Python <http://python.org/>`_ 3.7, 3.8, 3.9 or 3.10.

sudo apt-get install python-pip python-numpy python-scipy python-pip python-six python-tqdm


************
Installation
************
Expand All @@ -65,7 +64,6 @@ Installation

pip install elephant[extras]


To upgrade to a newer release use the ``--upgrade`` flag:

.. code-block:: sh
Expand Down Expand Up @@ -210,6 +208,37 @@ You can have one, both, or none installed in your system.
using your graphics card to perform computations may make the system
unresponsive until the compute program terminates.

.. _no-compile-spade:
***********
Resolving compilation issues
***********

Some modules in Elephant make use of C extensions to speed up computation.
However, those extensions need to be compiled before use. In some cases, this
causes problems. For example, the compiler on the current machine does not
fulfill the requirements for the extension, certain libraries are missing,
or no compiler is available at all.

In order to circumvent this problem, the following commands allow to avoid the
compilation for specific or for all C extensions.

.. tabs::

.. tab:: general
Use the following to install elephant without C extensions:

.. code-block:: sh

pip install elephant --install-option='--no-compile'

.. tab:: spade
To avoid compilation of the c++ extension ``fim.cpp`` used in :ref:`spade`, install the package with:

.. code-block:: sh

pip install elephant --install-option='--no-compile-spade'

In this case the pure python implementation of :ref:`spade` is still available.

************
Dependencies
Expand Down
2 changes: 2 additions & 0 deletions doc/reference/spade.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _spade:

==============================================
Spike Pattern Detection and Evaluation (SPADE)
==============================================
Expand Down
2 changes: 1 addition & 1 deletion elephant/cell_assembly_detection.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@

See Also
--------
elephant.spade.spade : advanced synchronous patterns detection
:ref:`spade` : advanced synchronous patterns detection

Examples
--------
Expand Down
31 changes: 21 additions & 10 deletions elephant/spade.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,36 @@

Visualization
-------------
Visualization of SPADE analysis is covered in Viziphant:
Visualization of SPADE analysis is covered in Viziphant e.g.:
:func:`viziphant.patterns.plot_patterns_statistics_all`

See Viziphant for more information:

https://viziphant.readthedocs.io/en/latest/modules.html


Notes
-----
This modules relies on the C++ implementation of the fp-growth algorithm developed by
Forian Porrmann (available at https://github.com/fporrmann/FPG). The module replaces
a more generic implementation of the algorithm by Christian Borgelt
(http://www.borgelt.net/pyfim.html) that was used in previous versions of Elephant.
If the module (fim.so) is not available in a precompiled format (currently Linux/Windows) or cannot
be compiled on a given system during install, SPADE will make use of a pure Python implementation
of the fast fca algorithm contained in `elephant/spade_src/fast_fca.py`, which is
This modules relies on the C++ implementation of the fp-growth
algorithm developed by Forian Porrmann (available at
https://github.com/fporrmann/FPG). The module replaces a more generic
implementation of the algorithm by Christian Borgelt (
http://www.borgelt.net/pyfim.html) that was used in previous versions of
Elephant.

If the module ``fim.so`` is not available in a precompiled format (
currently Linux/Windows) or cannot be compiled on a given system during
install, SPADE will make use of a pure Python implementation of the fast fca
algorithm contained in `elephant/spade_src/fast_fca.py`, which is
significantly slower.

See :ref:`no-compile-spade` on how to install elephant without compiling the
``fim.so`` module.


See Also
--------
elephant.cell_assembly_detection.cell_assembly_detection : another synchronous
:doc:`cell_assembly_detection`: another synchronous
patterns detection


Expand Down Expand Up @@ -971,7 +982,7 @@ def _rereference_to_last_spike(transactions, winlen):
neurons[idx] = attribute // winlen
bins[idx] = attribute % winlen

# rereference bins to last spike
# reference bins to last spike
bins = bins.max() - bins

# calculate converted transactions
Expand Down
101 changes: 95 additions & 6 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# -*- coding: utf-8 -*-
import os.path
import platform
import sys

from setuptools import setup, Extension
from setuptools.command.install import install
from setuptools.command.develop import develop

with open(os.path.join(os.path.dirname(__file__),
"elephant", "VERSION")) as version_file:
Expand All @@ -26,7 +29,9 @@
libraries=[],
extra_compile_args=[
'-DMODULE_NAME=fim', '-DUSE_OPENMP', '-DWITH_SIG_TERM',
'-Dfim_EXPORTS', '-fopenmp', '/std:c++17'])
'-Dfim_EXPORTS', '-fopenmp', '/std:c++17'],
optional=True
)
elif platform.system() == "Darwin":
fim_module = Extension(
name='elephant.spade_src.fim',
Expand All @@ -39,7 +44,9 @@
'-Dfim_EXPORTS', '-O3', '-pedantic', '-Wextra',
'-Weffc++', '-Wunused-result', '-Werror', '-Werror=return-type',
'-Xpreprocessor',
'-fopenmp', '-std=gnu++17'])
'-fopenmp', '-std=gnu++17'],
optional=True
)
elif platform.system() == "Linux":
fim_module = Extension(
name='elephant.spade_src.fim',
Expand All @@ -51,7 +58,9 @@
'-DMODULE_NAME=fim', '-DUSE_OPENMP', '-DWITH_SIG_TERM',
'-Dfim_EXPORTS', '-O3', '-pedantic', '-Wextra',
'-Weffc++', '-Wunused-result', '-Werror',
'-fopenmp', '-std=gnu++17'])
'-fopenmp', '-std=gnu++17'],
optional=True
)

setup_kwargs = {
"name": "elephant",
Expand Down Expand Up @@ -87,9 +96,89 @@
'Programming Language :: Python :: 3 :: Only',
'Topic :: Scientific/Engineering']
}
# do not compile external modules on darwin
if platform.system() in ["Windows", "Linux"]:
setup_kwargs["ext_modules"] = [fim_module]

# no compile options and corresponding extensions
options = {"--no-compile": None, "--no-compile-spade": fim_module}
# check if any option was specified
if not any([True for key in options.keys() if key in sys.argv]):
if platform.system() in ["Windows", "Linux"]:
setup_kwargs["ext_modules"] = [fim_module]
else: # ...any option was specified
# select extensions accordingly
extensions = [module for flag, module in options.items() if
flag not in sys.argv]
if None in extensions: # None indicates "--no-compile" not in sys.argv
extensions.remove(None)
setup_kwargs["ext_modules"] = extensions


class CommandMixin(object):
"""
This class acts as a superclass to integrate new commands in setuptools.
"""
user_options = [
('no-compile', None, 'do not compile any C++ extension'),
('no-compile-spade', None, 'do not compile spade related C++ extension') # noqa
]

def initialize_options(self):
"""
The method is responsible for setting default values for
all the options that the command supports.

Option dependencies should not be set here.
"""

super().initialize_options()
# Initialize options
self.no_compile_spade = None
self.no_compile = None

def finalize_options(self):
"""
Overriding a required abstract method.

This method is responsible for setting and checking the
final values and option dependencies for all the options
just before the method run is executed.

In practice, this is where the values are assigned and verified.
"""

super().finalize_options()

def run(self):
"""
Sets global which can later be used in setup.py to remove c-extensions
from setup call.
"""
# Use options
global no_compile_spade
global no_compile
no_compile_spade = self.no_compile_spade
no_compile = self.no_compile

super().run()


class InstallCommand(CommandMixin, install):
"""
This class extends setuptools.command.install class, adding user options.
"""
user_options = getattr(
install, 'user_options', []) + CommandMixin.user_options


class DevelopCommand(CommandMixin, develop):
"""
This class extends setuptools.command.develop class, adding user options.
"""
user_options = getattr(
develop, 'user_options', []) + CommandMixin.user_options


# add classes to setup-kwargs to add the user options
setup_kwargs['cmdclass'] = {'install': InstallCommand,
'develop': DevelopCommand}

setup(**setup_kwargs)