Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update blosc version #461

Closed
wants to merge 2 commits into from
Closed

Conversation

dstansby
Copy link
Contributor

Updates the blosc version to the latest 1.x release.

I think fixes #429?

TODO:

  • Unit tests and/or doctests in docstrings
  • Tests pass locally
  • Docstrings and API docs for any new/modified user-facing classes and functions
  • Changes documented in docs/release.rst
  • Docs build locally
  • GitHub Actions CI passes
  • Test coverage to 100% (Codecov passes)

@joshmoore
Copy link
Member

Thanks, @dstansby. Triggering workflows.

@codecov
Copy link

codecov bot commented Aug 30, 2023

Codecov Report

Merging #461 (2400183) into main (cb15543) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #461   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           56        55    -1     
  Lines         2242      2170   -72     
=========================================
- Hits          2242      2170   -72     

see 1 file with indirect coverage changes

@dstansby
Copy link
Contributor Author

😦 doesn't seem to be working. I'll take a look and see if I can work out why

@dstansby
Copy link
Contributor Author

I'm going to try bumping one minor release at a time from blosc 1.21.0 to 1.21.4 to help isolate the issue.

@dstansby
Copy link
Contributor Author

I'll close this and run CI in my own fork until it's fixed so you don't have to keep approving CI here - will re-open this PR ping if/when it's fixed!

@dstansby dstansby closed this Aug 30, 2023
@dstansby dstansby reopened this Aug 30, 2023
@dstansby
Copy link
Contributor Author

I've reverted to just doing the update from 1.21.0 > 1.21.1, and building seems to fail on some Python 3.8 jobs (see dstansby#1). I had a quick go a reproducing locally, but I can build the pacakge fine on Python 3.8, macos, arm64.

@dstansby
Copy link
Contributor Author

Yep, just seems to be the Linux Python 3.8 jobs that are failing 😢 Error seems to be:

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Ic-blosc/blosc -Ic-blosc/internal-complibs/lz4-1.9.3 -Ic-blosc/internal-complibs/zlib-1.2.11 -Ic-blosc/internal-complibs/zstd-1.5.0 -Ic-blosc/internal-complibs/zlib-1.2.11/msdos -Ic-blosc/internal-complibs/zlib-1.2.11/doc -Ic-blosc/internal-complibs/zlib-1.2.11/nintendods -Ic-blosc/internal-complibs/zlib-1.2.11/qnx -Ic-blosc/internal-complibs/zlib-1.2.11/test -Ic-blosc/internal-complibs/zlib-1.2.11/amiga -Ic-blosc/internal-complibs/zlib-1.2.11/examples -Ic-blosc/internal-complibs/zlib-1.2.11/old -Ic-blosc/internal-complibs/zlib-1.2.11/os400 -Ic-blosc/internal-complibs/zlib-1.2.11/win32 -Ic-blosc/internal-complibs/zlib-1.2.11/watcom -Ic-blosc/internal-complibs/zlib-1.2.11/contrib -Ic-blosc/internal-complibs/zstd-1.5.0/compress -Ic-blosc/internal-complibs/zstd-1.5.0/decompress -Ic-blosc/internal-complibs/zstd-1.5.0/dictBuilder -Ic-blosc/internal-complibs/zstd-1.5.0/deprecated -Ic-blosc/internal-complibs/zstd-1.5.0/legacy -Ic-blosc/internal-complibs/zstd-1.5.0/common -Ic-blosc/internal-complibs/zstd-1.5.0/dll -Ic-blosc/internal-complibs/zlib-1.2.11/old/os2 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/masmx86 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/asm686 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/ada -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/dotzlib -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/iostream2 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/pascal -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/infback9 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/untgz -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/iostream -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/inflate86 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/minizip -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/iostream3 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/gcc_gvmat64 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/blast -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/puff -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/masmx64 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/vstudio -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/amd64 -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/delphi -Ic-blosc/internal-complibs/zlib-1.2.11/contrib/testzlib -Ic-blosc/internal-complibs/zstd-1.5.0/dll/example -I/opt/_internal/cpython-3.8.17/include/python3.8 -c numcodecs/blosc.c -o build/temp.linux-x86_64-cpython-38/numcodecs/blosc.o -msse2 -mno-avx2 -DSHUFFLE_SSE2_ENABLED
        In file included from /opt/_internal/cpython-3.8.17/include/python3.8/Python.h:44,
                         from numcodecs/blosc.c:151:
        c-blosc/internal-complibs/zlib-1.2.11/contrib/minizip/crypt.h:35:53: error: unknown type name ‘z_crc_t’
           35 | static int decrypt_byte(unsigned long* pkeys, const z_crc_t* pcrc_32_tab)
              |                                                     ^~~~~~~
        c-blosc/internal-complibs/zlib-1.2.11/contrib/minizip/crypt.h:48:51: error: unknown type name ‘z_crc_t’
           48 | static int update_keys(unsigned long* pkeys,const z_crc_t* pcrc_32_tab,int c)
              |                                                   ^~~~~~~
        c-blosc/internal-complibs/zlib-1.2.11/contrib/minizip/crypt.h:65:69: error: unknown type name ‘z_crc_t’
           65 | static void init_keys(const char* passwd,unsigned long* pkeys,const z_crc_t* pcrc_32_tab)
              |                                                                     ^~~~~~~

@dstansby dstansby marked this pull request as draft October 31, 2023 10:56
@joshmoore joshmoore mentioned this pull request Dec 20, 2023
7 tasks
@mkitti
Copy link
Contributor

mkitti commented Dec 21, 2023

#500 supersedes this with all tests passing.

  • It upgrades to c-blosc 1.21.5 (including Zstandard 1.5.5)
  • The Python 3.8 issue is addressed by not including the minizip directory
  • Assembly files are compiled on x86_64

@dstansby dstansby closed this Jan 11, 2024
@dstansby dstansby deleted the bump-blosc branch January 11, 2024 11:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Upgrade to ZSTD 1.5.5 due to potential corruption
3 participants