Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for Python 3.10 #865

Merged
merged 14 commits into from
Nov 19, 2021
Merged

Add support for Python 3.10 #865

merged 14 commits into from
Nov 19, 2021

Conversation

nsoranzo
Copy link
Contributor

Also:

  • Cleanup pre-3.6 code with pyupgrade
  • Add requirement for >=3.6 to setup.cfg

@nsoranzo
Add support for Python 3.10
aa82b53 
Also:
- Cleanup pre-3.6 code with pyupgrade
- Add requirement for `>=3.6` to `setup.cfg`
@wasade
Copy link
Member

wasade commented Nov 18, 2021

Thank you!!! This is very helpful! I'm 👍 assuming CI is green, I just approved the deploy

@nsoranzo
Copy link
Contributor Author

Thanks @wasade ! For reference, I've added the minimal patch to support Python 3.10 to conda-forge/biom-format-feedstock#17 .

@wasade
Copy link
Member

wasade commented Nov 18, 2021

I saw :) and thank you for that as well :)

@wasade
Copy link
Member

wasade commented Nov 18, 2021

@nsoranzo any chance you could modify gh actions to include 3.7-3.10? (https://github.com/biocore/biom-format/blob/master/.github/workflows/python-package-conda.yml#L49) Just saw it was pinned, don't recall why, and i'm curious what happens here. If failures do arise, I'll still merge this PR as it's green, and I'll open a separate gh issue to resolve them.

@nsoranzo
Copy link
Contributor Author

I normally find that just min and max supported versions are sufficient (i.e. ['3.6', '3.10']), but up to you!

@wasade
Copy link
Member

wasade commented Nov 18, 2021

Good point, and that's also totally fine, thank you!

@nsoranzo
Copy link
Contributor Author

nsoranzo commented Nov 18, 2021 •
edited
Loading

The 3.10 job is failing I think because there is no scikit-bio conda package for Python 3.10 yet.

@wasade
Copy link
Member

wasade commented Nov 18, 2021

Okay, thanks. That would explain it, and I know the team is working on a new release. I don't think that is a real dependency, or at least I'm not finding imports for it anymore -- want to remove it from the gh actions conda install? (i'd suggest/commit but gh isn't letting me do so in the pr)

@nsoranzo
Copy link
Contributor Author

It was added in #859 and it's still used in biom/table.py and biom/tests/test_table.py (grep -ir skbio).

I'd suggest/commit but gh isn't letting me do so in the PR

Weird, "Allow edits by maintainers" is enabled in the PR.

@wasade
Copy link
Member

wasade commented Nov 18, 2021

Clearly both my gh and grep foo are failing.

it's an optional dependency, and only needed for doc build which is done via py36. Removed, let's see what happens

@wasade
Copy link
Member

wasade commented Nov 18, 2021

Seems like some other dependency, not skbio, is being problematic:

https://github.com/biocore/biom-format/runs/4255448804?check_suite_focus=true

@nsoranzo
Copy link
Contributor Author

You are currently running flake8 twice in the GitHub workflow, once in the lint job and once (per Python version) in the build job.

@wasade
Copy link
Member

wasade commented Nov 18, 2021

Could be, this was the first project we migrated to gh-actions from travis, and the config is still a little rough around the edges

@wasade
Copy link
Member

wasade commented Nov 18, 2021

I'm going to restore the dependencies for right now, it doesn't seem like an skbio issue but rather somewhere else. I'll issue a separate PR to remove six/future as they're no longer necessary

@nsoranzo
Copy link
Contributor Author

I'll issue a separate PR to remove six/future as they're no longer necessary

I've now removed those here, I think it's a good fit for this PR :)
Also, I've specified that conda should use conda-forge, I don't think default has so many Python 3.10 packages.

@wasade
Copy link
Member

wasade commented Nov 18, 2021

You're quickly becoming my favorite person

@wasade
Copy link
Member

wasade commented Nov 18, 2021

Unfortunately, those are real. Possible the h5py build in conda-forge is unusual?

@nsoranzo
Fix test_from_hdf5_custom_parsers on h5py >=3.0
62a87f7
@nsoranzo
Fix "VLEN strings do not support embedded NULLs"
f3d5b50 
Fix the following traceback with h5py 3.1.0 :

```
test_table.py:1025: in test_to_hdf5_missing_metadata_sample
    t.to_hdf5(h5, 'tests')
../table.py:4563: in to_hdf5
    formatter[category](grp, category, md, compression)
../table.py:315: in general_formatter
    compression=compression)
/usr/share/miniconda/envs/env_name/lib/python3.6/site-packages/h5py/_hl/group.py:148: in create_dataset
    dsid = dataset.make_new_dset(group, shape, dtype, data, name, **kwds)
/usr/share/miniconda/envs/env_name/lib/python3.6/site-packages/h5py/_hl/dataset.py:140: in make_new_dset
    dset_id.write(h5s.ALL, h5s.ALL, data)
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
    ???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
    ???
h5py/h5d.pyx:232: in h5py.h5d.DatasetID.write
    ???
h5py/_proxy.pyx:145: in h5py._proxy.dset_rw
    ???
h5py/_conv.pyx:443: in h5py._conv.str2vlen
    ???
h5py/_conv.pyx:94: in h5py._conv.generic_converter
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   ValueError: VLEN strings do not support embedded NULLs
```
@nsoranzo
Quotes
346a388
@nsoranzo
Copy link
Contributor Author

Unfortunately, those are real. Possible the h5py build in conda-forge is unusual?

@wasade The conda-forge build is not unusual, but when using conda-forge h5py 3.1.0 gets installed instead of 2.10, and this caused the issues. I've fixed them in 2 new commits, hope they make sense.

@nsoranzo
Copy link
Contributor Author

Looks like nose is needed for @npt.dec.skipif(), should that be replaced by the pytest equivalent?

@wasade
Copy link
Member

wasade commented Nov 19, 2021

This is hitting some scope creep :/ if you're able that would be nice, thanks. No preference for pytest vs unittest.skipIf

@nsoranzo
Copy link
Contributor Author

All green 🎉

@wasade
Copy link
Member

wasade commented Nov 19, 2021

Fantastic!!! Thank you so much! I'll look at a release in the coming weeks so this can be included quickly :)

@wasade wasade merged commit fa21afd into biocore:master Nov 19, 2021
@nsoranzo nsoranzo deleted the python3.10 branch November 19, 2021 22:31
@wasade wasade mentioned this pull request Mar 25, 2022
Closed
@Vlasovets
Copy link

Vlasovets commented May 2, 2023 •
edited
Loading

hi @nsoranzo!

I'm experiencing an installation problem using Docker and Python 3.10.
I was able to successfully install biom-format on Google Colab using pip and python 3.10.11, but with Docker it surprisingly failed, may I ask you to navigate to a solution?

I attach the output describing the error (if it is ok), I could also paste it here, but it is rather long.
Otherwise, it is possible to reproduce the error, running the building the following Docker image:

FROM quay.io/qiime2/core:2023.2 RUN apt-get update && apt-get -y install python3.10 WORKDIR /app COPY requirements.txt . ### biom-format RUN python3.10 -m pip install -r requirements.txt

Thank you in advance!
error.txt

@wasade
Copy link
Member

wasade commented May 2, 2023

Hi @Vlasovets, it looks like the environment is missing a compiler, see error: command 'gcc' failed: No such file or directory near the end of the error output

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nsoranzo @wasade @Vlasovets