Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem installing polars_1xx branch, possible issue through pybigtools [BUG] #197

Open
ckv2115147 opened this issue Dec 30, 2024 · 2 comments
Open

Problem installing polars_1xx branch, possible issue through pybigtools [BUG] #197

ckv2115147 opened this issue Dec 30, 2024 · 2 comments

Comments

@ckv2115147
Copy link

I want to install the polars_1xx branch to process a large dataset >100k cells, but am having problems installing pycisTopic. It seems the issue may be from installing pybigtools.

I followed these steps to install the polars_1xx branch in a conda env:

$ conda create --name pycistopicpolars python=3.11
$ conda install -n pycistopicpolars -c conda-forge rust=1.82.0
$ git clone https://github.com/aertslab/pycisTopic.git
$ git checkout polars_1xx
$ pip install .

But got this error:

  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting numpy<2,>=1.20.3 (from pycisTopic==2.0a0)
  Using cached numpy-1.26.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Collecting pandas==1.5 (from pycisTopic==2.0a0)  
  Using cached pandas-1.5.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)
Collecting polars>=1 (from pycisTopic==2.0a0)
  Using cached polars-1.18.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (14 kB)
Collecting pyarrow>=8.0.0 (from pycisTopic==2.0a0)
  Using cached pyarrow-18.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.3 kB)
Collecting pyranges<0.0.128 (from pycisTopic==2.0a0)
  Using cached pyranges-0.0.127-py3-none-any.whl.metadata (15 kB)
Collecting numba (from pycisTopic==2.0a0)
  Using cached numba-0.60.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.7 kB)
Collecting ray (from pycisTopic==2.0a0)
  Using cached ray-2.40.0-cp311-cp311-manylinux2014_x86_64.whl.metadata (17 kB)
Collecting scatac_fragment_tools<0.2.0 (from pycisTopic==2.0a0)
  Using cached scatac_fragment_tools-0.1.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.8 kB)
Collecting scikit-learn (from pycisTopic==2.0a0) 
  Using cached scikit_learn-1.6.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)
Collecting lda (from pycisTopic==2.0a0)
  Using cached lda-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.6 kB)
Collecting matplotlib<3.7 (from pycisTopic==2.0a0)
  Using cached matplotlib-3.6.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.8 kB)
Collecting tmtoolkit (from pycisTopic==2.0a0)
  Using cached tmtoolkit-0.12.0-py3-none-any.whl.metadata (14 kB)
Collecting gensim (from pycisTopic==2.0a0)
  Using cached gensim-4.3.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.1 kB)
Collecting macs2 (from pycisTopic==2.0a0)
  Using cached MACS2-2.2.9.1-cp311-cp311-linux_x86_64.whl
Collecting pybiomart (from pycisTopic==2.0a0)
  Using cached pybiomart-0.2.0-py3-none-any.whl.metadata (1.1 kB)
Collecting pyscenic (from pycisTopic==2.0a0)
  Using cached pyscenic-0.12.1-py3-none-any.whl.metadata (9.8 kB)
Collecting scrublet (from pycisTopic==2.0a0)
  Using cached scrublet-0.2.3-py3-none-any.whl.metadata (3.1 kB)
Collecting harmonypy (from pycisTopic==2.0a0)
  Using cached harmonypy-0.0.10-py3-none-any.whl.metadata (3.9 kB)
Collecting igraph (from pycisTopic==2.0a0)
  Using cached igraph-0.11.8-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting leidenalg (from pycisTopic==2.0a0)
  Using cached leidenalg-0.10.2-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (10 kB)
Collecting seaborn (from pycisTopic==2.0a0)
  Using cached seaborn-0.13.2-py3-none-any.whl.metadata (5.4 kB)
Collecting adjustText (from pycisTopic==2.0a0)   
  Using cached adjustText-1.3.0-py3-none-any.whl.metadata (3.1 kB)
Collecting statsmodels (from pycisTopic==2.0a0)  
  Using cached statsmodels-0.14.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.2 kB)
Collecting scanpy (from pycisTopic==2.0a0)
  Using cached scanpy-1.10.4-py3-none-any.whl.metadata (9.3 kB)
Collecting scanorama (from pycisTopic==2.0a0)
  Using cached scanorama-1.7.4-py3-none-any.whl.metadata (525 bytes)
Collecting bbknn (from pycisTopic==2.0a0)
  Using cached bbknn-1.6.0-py3-none-any.whl.metadata (8.2 kB)
Collecting loomxpy (from pycisTopic==2.0a0)
  Using cached loomxpy-0.4.2-py3-none-any.whl.metadata (1.6 kB)
Collecting bs4 (from pycisTopic==2.0a0)
  Using cached bs4-0.0.2-py2.py3-none-any.whl.metadata (411 bytes)
Collecting lxml (from pycisTopic==2.0a0)
  Using cached lxml-5.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
Collecting python-dateutil>=2.8.1 (from pandas==1.5->pycisTopic==2.0a0)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas==1.5->pycisTopic==2.0a0)
  Using cached pytz-2024.2-py2.py3-none-any.whl.metadata (22 kB)
Collecting contourpy>=1.0.1 (from matplotlib<3.7->pycisTopic==2.0a0)
  Using cached contourpy-1.3.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.4 kB)
Collecting cycler>=0.10 (from matplotlib<3.7->pycisTopic==2.0a0)
  Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
Collecting fonttools>=4.22.0 (from matplotlib<3.7->pycisTopic==2.0a0)
  Using cached fonttools-4.55.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (165 kB)
Collecting kiwisolver>=1.0.1 (from matplotlib<3.7->pycisTopic==2.0a0)
  Using cached kiwisolver-1.4.8-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.2 kB)
Collecting packaging>=20.0 (from matplotlib<3.7->pycisTopic==2.0a0)
  Using cached packaging-24.2-py3-none-any.whl.metadata (3.2 kB)
Collecting pillow>=6.2.0 (from matplotlib<3.7->pycisTopic==2.0a0)
  Using cached pillow-11.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (9.1 kB)
Collecting pyparsing>=2.2.1 (from matplotlib<3.7->pycisTopic==2.0a0)
  Using cached pyparsing-3.2.0-py3-none-any.whl.metadata (5.0 kB)
Collecting ncls>=0.0.63 (from pyranges<0.0.128->pycisTopic==2.0a0)
  Using cached ncls-0.0.68-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.6 kB)
Collecting tabulate (from pyranges<0.0.128->pycisTopic==2.0a0)
  Using cached tabulate-0.9.0-py3-none-any.whl.metadata (34 kB)
Collecting sorted-nearest>=0.0.33 (from pyranges<0.0.128->pycisTopic==2.0a0)
  Using cached sorted_nearest-0.0.39-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (964 bytes)
Collecting natsort (from pyranges<0.0.128->pycisTopic==2.0a0)
  Using cached natsort-8.4.0-py3-none-any.whl.metadata (21 kB)
Collecting pybigwig (from scatac_fragment_tools<0.2.0->pycisTopic==2.0a0)
  Using cached pyBigWig-0.3.23-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Collecting pybigtools>=0.2.0 (from scatac_fragment_tools<0.2.0->pycisTopic==2.0a0)
  Using cached pybigtools-0.2.3.tar.gz (9.5 MB)  
  Installing build dependencies ... done
  Getting requirements to build wheel ... done   
  Preparing metadata (pyproject.toml) ... error  
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
          Updating crates.io index
           Locking 15 packages to latest compatible versions
            Adding base64 v0.21.7 (latest: v0.22.1)
          Updating bigtools v0.3.1-dev (/work/tmp/pip-install-0jw93rsr/pybigtools_1227f15a78884eeebc41971527c93b01/bigtools) -> v0.5.4
            Adding getrandom v0.2.10 (latest: v0.2.15)
            Adding index_list v0.2.15
          Updating pybigtools v0.1.0 (/work/tmp/pip-install-0jw93rsr/pybigtools_1227f15a78884eeebc41971527c93b01/pybigtools) -> v0.2.3
            Adding ring v0.17.3 (latest: v0.17.8)
            Adding rustls v0.21.12 (latest: v0.23.20)
            Adding rustls-native-certs v0.6.3 (latest: v0.8.1)
            Adding rustls-pemfile v1.0.4 (latest: v2.2.0)
            Adding rustls-webpki v0.101.7 (latest: v0.102.8)
            Adding sct v0.7.1
            Adding spin v0.9.8
            Adding untrusted v0.9.0
            Adding wasi v0.11.0+wasi-snapshot-preview1 (latest: v0.13.3+wasi-0.2.2)
            Adding webpki v0.22.4
      � maturin failed
        Caused by: `project.version` field is required in pyproject.toml unless it is present in the `project.dynamic` list
      Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/work/tmp/pip-modern-metadata-pnst2ea0', '--interpreter', '/miniconda/envs/pycistopicpolars/bin/python3.11']' returned non-zero exit status 1.
      Checking for Rust toolchain....
      Running `maturin pep517 write-dist-info --metadata-directory /work/tmp/pip-modern-metadata-pnst2ea0 --interpreter /miniconda/envs/pycistopicpolars/bin/python3.11`
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

Version:

  • Python: 3.11.10
  • gcc 11.3.0

I think the bug is related to installing scatac_fragment_tools, which in turn is trying to install pybigtools, which is where the installation stalls. Would appreciate any help with this!
@ghuls
Copy link
Member

ghuls commented Dec 30, 2024

Looks like there is a breaking change in maturin downgrade to maturin 1.7.8 for now:
PyO3/maturin#2416

@ghuls
Copy link
Member

ghuls commented Dec 30, 2024 •
edited
Loading

Or you can clone the bigtools repo and apply this patch and install pybigtools from the patched version: jackh726/bigtools#68

Update: Building from bigtools master branch should work now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ghuls @ckv2115147