python3Packages.sourmash: init at 4.8.3 #230596

luizirber · 2023-05-07T23:56:52Z

Description of changes

sourmash is a bioinformatics tool for quickly searching, comparing, and analyzing genomic and metagenomic data sets.

Homepage: https://sourmash.bio
GitHub: https://github.com/sourmash-bio/sourmash

Things done

Built on platform(s)
- x86_64-linux
- aarch64-linux
- x86_64-darwin
- aarch64-darwin
For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
Tested, as applicable:
- NixOS test(s) (look inside nixos/tests)
- and/or package tests
- or, for functions and "core" functionality, tests in lib/tests or pkgs/test
- made sure NixOS tests are linked to the relevant packages
Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)
- [-] (Package updates) Added a release notes entry if the change is major or breaking
- [-] (Module updates) Added a release notes entry if the change is significant
- [-] (Module addition) Added a release notes entry if adding a new NixOS module
Fits CONTRIBUTING.md.

nixos-discourse · 2023-05-11T03:50:01Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/2192

pkgs/development/python-modules/sourmash/default.nix

SuperSandro2000 · 2023-05-12T12:58:51Z

@ofborg build python310Packages.sourmash

pkgs/development/python-modules/sourmash/default.nix

SuperSandro2000 · 2023-05-17T14:44:05Z

one test fails:

=================================== FAILURES ===================================
_ test_gather_check_scaled_bounds_less_than_minimum[--no-prefetch---no-linear] _
[gw58] linux -- Python 3.10.11 /nix/store/95cxzy2hpizr23343b8bskl4yacf4b3l-python3-3.10.11/bin/python3.10
runtmp = <sourmash_tst_utils.RunnerContext object at 0x7ffbe29cd0f0>
prefetch_gather = '--no-prefetch', linear_gather = '--no-linear'
    def test_gather_check_scaled_bounds_less_than_minimum(runtmp, prefetch_gather, linear_gather):
        testdata_glob = utils.get_test_data('gather/GCF*.sig')
        testdata_sigs = glob.glob(testdata_glob)
    
        query_sig = utils.get_test_data('gather/combined.sig')
    
        with pytest.raises(SourmashCommandFailed):
            runtmp.sourmash('gather', query_sig, prefetch_gather, linear_gather, 'gcf_all', '-k', '21', '--scaled', '50', '--threshold-bp', '50000')
    
>       assert "WARNING: scaled value should be >= 100. Continuing anyway." in runtmp.last_result.err
E       assert 'WARNING: scaled value should be >= 100. Continuing anyway.' in '\r\x1b[Kselecting specified query k=21\n\r\x1b[Kloaded query: -... (k=21, DNA)\n\r\x1b[Kdownsampling query from scale...path.py", line 76, in join\n    a = os.fspath(a)\nTypeError: expected str, bytes or os.PathLike object, not NoneType\n'
E        +  where '\r\x1b[Kselecting specified query k=21\n\r\x1b[Kloaded query: -... (k=21, DNA)\n\r\x1b[Kdownsampling query from scale...path.py", line 76, in join\n    a = os.fspath(a)\nTypeError: expected str, bytes or os.PathLike object, not NoneType\n' = ScriptResults(status=-1, out='', err='\r\x1b[Kselecting specified query k=21\n\r\x1b[Kloaded query: -... (k=21, DNA)\n...ath.py", line 76, in join\n    a = os.fspath(a)\nTypeError: expected str, bytes or os.PathLike object, not NoneType\n').err
E        +    where ScriptResults(status=-1, out='', err='\r\x1b[Kselecting specified query k=21\n\r\x1b[Kloaded query: -... (k=21, DNA)\n...ath.py", line 76, in join\n    a = os.fspath(a)\nTypeError: expected str, bytes or os.PathLike object, not NoneType\n') = <sourmash_tst_utils.RunnerContext object at 0x7ffbe29cd0f0>.last_result
tests/test_sourmash.py:4417: AssertionError
----------------------------- Captured stdout call -----------------------------
running: sourmash in: /build/sourmashtest_33f6e022
arguments ['sourmash', 'gather', '/build/sourmash-4.8.2/tests/test-data/gather/combined.sig', '--no-prefetch', '--no-linear', 'gcf_all', '-k', '21', '--scaled', '50', '--threshold-bp', '50000']
=============================== warnings summary ===============================

doronbehar · 2023-06-26T10:22:25Z

Result of nixpkgs-review pr 230596 run on x86_64-linux 1

4 packages failed to build:

python310Packages.sourmash
python310Packages.sourmash.dist
python311Packages.sourmash
python311Packages.sourmash.dist

luizirber · 2023-06-27T00:45:57Z

I'm very confused with the current build error... Before it was related to tests, but now it is because a weird mode 130 is being inserted in the ffi.dlopen, but that doesn't happen by regular Python packaging tooling (nor by running nix build github:sourmash-bio/sourmash# ; cat result/lib/python3.10/site-packages/sourmash/_lowlevel/__init__.py, the flake-based package building I have in the upstream repo).

$ nix build .#python311Packages.sourmash
error: builder for '/nix/store/ha2j8l482pn0va7wkddb96vasvdmab4a-python3.11-sourmash-4.8.2.drv' failed with exit code 1;
       last 10 log lines:
       >   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
       >   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
       >   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
       >   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
       >   File "/nix/store/qgv5ldklwg064vzdal7djzc1wv85f2l2-python3.11-sourmash-4.8.2/lib/python3.11/site-packages/sourmash/__init__.py", line 33, in <module>
       >     from ._lowlevel import ffi, lib
       >   File "/nix/store/qgv5ldklwg064vzdal7djzc1wv85f2l2-python3.11-sourmash-4.8.2/lib/python3.11/site-packages/sourmash/_lowlevel.py", line 7, in <module>
       >     lib = ffi.dlopen(os.path.join(os.path.dirname(__file__), '_lowlevel__lib.so'), 130)
       >           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       > OSError: cannot load library '/nix/store/qgv5ldklwg064vzdal7djzc1wv85f2l2-python3.11-sourmash-4.8.2/lib/python3.11/site-packages/sourmash/_lowlevel__lib.so': invalid mode parameter
       For full logs, run 'nix log /nix/store/ha2j8l482pn0va7wkddb96vasvdmab4a-python3.11-sourmash-4.8.2.drv'.

Is anyone aware of a Python hook that could be changing that? Not sure how to fix this...

natsukium

While this PR is open, maturin of nixpkgs is reached 1.0.0 in #230799.
This is why the build failed.
I can build it by applying these two patches.
sourmash-bio/sourmash@9de27b6
sourmash-bio/sourmash@406b945

luizirber · 2023-08-03T22:18:21Z

While this PR is open, maturin of nixpkgs is reached 1.0.0 in #230799. This is why the build failed. I can build it by applying these two patches. sourmash-bio/sourmash@9de27b6 sourmash-bio/sourmash@406b945

Indeed, thanks @natsukium! I will do a new release of the upstream project and update the PyPI download link

natsukium · 2023-08-18T02:46:30Z

pkgs/development/python-modules/sourmash/default.nix

+
+  src = fetchPypi {
+    inherit pname version;
+    sha256 = "sha256-LIMpL9cLafytRFyPam/FBNi757j1v6o1FG/K2JknDQY=";


Suggested change

sha256 = "sha256-LIMpL9cLafytRFyPam/FBNi757j1v6o1FG/K2JknDQY=";

hash = "sha256-LIMpL9cLafytRFyPam/FBNi757j1v6o1FG/K2JknDQY=";

pkgs/development/python-modules/sourmash/default.nix

natsukium · 2023-08-18T02:53:34Z

@GrahamcOfBorg build python310Packages.sourmash

natsukium · 2023-08-19T01:54:46Z

Result of nixpkgs-review pr 230596 run on x86_64-linux 1

4 packages built:

python310Packages.sourmash
python310Packages.sourmash.dist
python311Packages.sourmash
python311Packages.sourmash.dist

natsukium

LGTM

(There are too many things happening in #2230, so I'll open a couple of PRs to simplify it) - Bring tox config as close as possible to `tox 4`, but keep using `tox 3` until [`tox-conda` is compatible with `tox 4`](tox-dev/tox-conda#156). Ended up adding an explicit `'tox>=3.27,<4'` to the conda dev env instructions to get the right version, and remove/replace deprecated options from `tox.ini` - Bump `MACOSX_DEPLOYMENT_TARGET` to `11.0` consistently for maturin and CI. - Remove Python installation for Rust tests in CI (not used) - Clean up `flake.nix` and avoid `naersk`, prefer regular `nixpkgs` tooling now that NixOS/nixpkgs#230596 is merged and we can keep them in sync - Move `.ci/install_cargo.sh` into `cibuildwheel` configuration, set up extra tools needed for `bindgen`/`cbindgen` (libclang and env vars)

github-actions bot added the 6.topic: python label May 7, 2023

ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels May 8, 2023

luizirber force-pushed the lirber/sourmash branch from c21303f to 6c2a916 Compare May 11, 2023 01:57

SuperSandro2000 reviewed May 11, 2023

View reviewed changes

pkgs/development/python-modules/sourmash/default.nix Outdated Show resolved Hide resolved

luizirber force-pushed the lirber/sourmash branch from 6c2a916 to eac5eab Compare May 11, 2023 21:34

SuperSandro2000 reviewed May 12, 2023

View reviewed changes

pkgs/development/python-modules/sourmash/default.nix Outdated Show resolved Hide resolved

luizirber force-pushed the lirber/sourmash branch from eac5eab to 200e5b8 Compare May 12, 2023 19:20

luizirber requested a review from SuperSandro2000 May 12, 2023 21:08

luizirber force-pushed the lirber/sourmash branch from 200e5b8 to 44167e6 Compare June 27, 2023 00:39

natsukium reviewed Jul 19, 2023

View reviewed changes

luizirber changed the title ~~python3Packages.sourmash: init at 4.8.2~~ python3Packages.sourmash: init at 4.8.3 Aug 17, 2023

luizirber force-pushed the lirber/sourmash branch from 44167e6 to 6e8fece Compare August 17, 2023 17:57

luizirber requested a review from natsukium August 17, 2023 18:53

luizirber mentioned this pull request Aug 17, 2023

sourmash v4.8.3 release - checklist and release notes sourmash-bio/sourmash#2718

Closed

13 tasks

natsukium reviewed Aug 18, 2023

View reviewed changes

python3Packages.sourmash: init at 4.8.3

e17a9b7

luizirber force-pushed the lirber/sourmash branch from 6e8fece to e17a9b7 Compare August 19, 2023 01:39

luizirber requested a review from natsukium August 19, 2023 01:40

natsukium approved these changes Aug 19, 2023

View reviewed changes

natsukium merged commit 30683ca into NixOS:master Aug 19, 2023

luizirber mentioned this pull request Aug 19, 2023

build: updates spun off mastiff PR sourmash-bio/sourmash#2723

Merged

luizirber deleted the lirber/sourmash branch August 28, 2023 00:18

luizirber mentioned this pull request Sep 22, 2023

sourmash: add v4.8.4 spack/spack#40062

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python3Packages.sourmash: init at 4.8.3 #230596

python3Packages.sourmash: init at 4.8.3 #230596

luizirber commented May 7, 2023

nixos-discourse commented May 11, 2023

SuperSandro2000 commented May 12, 2023 •

edited

Loading

SuperSandro2000 commented May 17, 2023

doronbehar commented Jun 26, 2023

luizirber commented Jun 27, 2023

natsukium left a comment

luizirber commented Aug 3, 2023

natsukium Aug 18, 2023

natsukium commented Aug 18, 2023 •

edited

Loading

natsukium commented Aug 19, 2023

natsukium left a comment

	sha256 = "sha256-LIMpL9cLafytRFyPam/FBNi757j1v6o1FG/K2JknDQY=";
	hash = "sha256-LIMpL9cLafytRFyPam/FBNi757j1v6o1FG/K2JknDQY=";

python3Packages.sourmash: init at 4.8.3 #230596

python3Packages.sourmash: init at 4.8.3 #230596

Conversation

luizirber commented May 7, 2023

Description of changes

Things done

nixos-discourse commented May 11, 2023

SuperSandro2000 commented May 12, 2023 • edited Loading

SuperSandro2000 commented May 17, 2023

doronbehar commented Jun 26, 2023

luizirber commented Jun 27, 2023

natsukium left a comment

Choose a reason for hiding this comment

luizirber commented Aug 3, 2023

natsukium Aug 18, 2023

Choose a reason for hiding this comment

natsukium commented Aug 18, 2023 • edited Loading

natsukium commented Aug 19, 2023

natsukium left a comment

Choose a reason for hiding this comment

SuperSandro2000 commented May 12, 2023 •

edited

Loading

natsukium commented Aug 18, 2023 •

edited

Loading