Python wrapper for the C-Blosc2 library.
Project description
A Python wrapper for the extremely fast Blosc2 compression library
- Author:
The Blosc development team
- Contact:
- Github:
- Actions:
- PyPi:
- NumFOCUS:
- Code of Conduct:
What it is
C-Blosc2 is the new major version of C-Blosc, and is backward compatible with both the C-Blosc1 API and its in-memory format.
Python-Blosc2 is a Python package that wraps C-Blosc2, the newest version of the Blosc compressor. Currently Python-Blosc2 already reproduces the API of Python-Blosc, so it can be used as a drop-in replacement. However, there are a few exceptions for a full compatibility.
In addition, Python-Blosc2 aims to leverage the new C-Blosc2 API so as to support super-chunks, multi-dimensional arrays (NDArray), serialization and other bells and whistles introduced in C-Blosc2. Although this is always and endless process, we are have almost catched up with the full C-Blosc2 potential (with the convenience of Python :-).
Note: Python-Blosc2 is meant to be backward compatible with Python-Blosc data. That means that it can read data generated with Python-Blosc, but the opposite is not true (i.e. there is no forward compatibility).
NDArray: an N-Dimensional store
One of the latest and more exciting additions in Python-Blosc2 is the NDArray object. It can write and read n-dimensional datasets in an extremely efficient way thanks to a n-dim 2-level partitioning, allowing to slice and dice arbitrary large and compressed data in a more fine-grained way:
To wet you appetite, here it is how the NDArray object performs on getting slices orthogonal to the different axis of a 4-dim dataset:
We have blogged about this: https://www.blosc.org/posts/blosc2-ndim-intro
We also have a explanatory video on why the double partition is useful (~2 min).
Installing
Blosc is now offering Python wheels for the main OS (Win, Mac and Linux) and platforms. You can install binary packages from PyPi using pip:
pip install blosc2
Documentation
The documentation is here:
https://blosc.org/python-blosc2/python-blosc2.html
Also, some examples are available on:
Building from sources
python-blosc2 comes with the C-Blosc2 sources with it and can be built in-place:
git clone https://github.com/Blosc/python-blosc2/
cd python-blosc2
git submodule update --init --recursive
python -m pip install -r requirements-build.txt
python setup.py build_ext --inplace
That’s all. You can proceed with testing section now.
Testing
After compiling, you can quickly check that the package is sane by running the tests:
python -m pip install -r requirements-tests.txt
python -m pytest (add -v for verbose mode)
Benchmarking
If curious, you may want to run a small benchmark that compares a plain NumPy array copy against compression through different compressors in your Blosc build:
PYTHONPATH=. python bench/pack_compress.py
License
The software is licenses under a 3-Clause BSD license. A copy of the python-blosc2 license can be found in LICENSE.txt.
Mailing list
Discussion about this module is welcome in the Blosc list:
Please follow @Blosc2 to get informed about the latest developments.
Enjoy data!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for blosc2-2.1.1-cp311-cp311-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f684b4a6ee21b035c48c8d722f0811fedd4bac7a8c675c24822213a2a4970b77 |
|
MD5 | ffa2a30fac5ee33ab860da3afa290e71 |
|
BLAKE2b-256 | 31075ff7b82ec53e5d5005c7d901c5a1dbde1c4d917dc7da5da7ee8a0b2f03cd |
Hashes for blosc2-2.1.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ef88da66bcafb790cd95154bb74798624a8d2c2032ef1391048501b60942cc6f |
|
MD5 | 8eb0af58b56d22cd14e20fb773473dae |
|
BLAKE2b-256 | 115699120b57fb7f22f40b99edb0642ad3e8a00919da4d210f562b21a68baf14 |
Hashes for blosc2-2.1.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 668329c3cd96ca331c04c17ccccdba5b15906a37da68a2d856ae3dc78b7bb743 |
|
MD5 | 0252ad5f422101f216e0f4e5aeda6913 |
|
BLAKE2b-256 | 4ad8d7b98b186c22c414d0ecf056f4da6545a58412210424fbc091c64485e56c |
Hashes for blosc2-2.1.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 156dbdb3af410895ee5af5dc1e43e18a4fa3e2b7116d22f04cc409a629f3e221 |
|
MD5 | b174a7562df4e4e31289c13de3a4775f |
|
BLAKE2b-256 | 9eea6fb20023b789414b460db77c545d831d2b82c92089f833f1f0d0f431313b |
Hashes for blosc2-2.1.1-cp311-cp311-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5a0f4325574cf92c4bd978dc7d51a7d69f0b4d0d85792f3582bccb3bb49a111d |
|
MD5 | 74141d0f9f4fb5c8fd5f1f99502b42ef |
|
BLAKE2b-256 | d23bf9bff020d84533cc47db7f2ed007dd2640622f8ae4c0ff4f800e80375cb4 |
Hashes for blosc2-2.1.1-cp310-cp310-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd6110a98d996d0a039ec2ff3e550dd0841b2ebac2234c5f7cb607c6c0b56e97 |
|
MD5 | c5b5e491d2152d8f6a81cc5f0fee36df |
|
BLAKE2b-256 | 2d62a5a885ef3378263340baa056a924670dba1efce410048cfd03c8b208f615 |
Hashes for blosc2-2.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | baaca702364a7b7b512e831302d7eb68858a528c3eabbb963f4134a465237b5e |
|
MD5 | f70d5451b6ee79854f53424caccc056d |
|
BLAKE2b-256 | c1935d1d3c3d584e2416fab605a8d7c7c9cbd569d169c4ad280c9745a1fbbe68 |
Hashes for blosc2-2.1.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | dee0d78d5c49862611107e5cbe37dadd0e226dd54fbc55ea52453e67c74a262d |
|
MD5 | 214be751598386155e60dc49a2eed4eb |
|
BLAKE2b-256 | 41df41e2ed0ec464b6591db9788b4efa069608159d2f6b3f8de1d1cea60ac9c8 |
Hashes for blosc2-2.1.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7dcaaad67f099f3952da34faba98129c8905657854ce69acc01692869631bedc |
|
MD5 | e8228d84d902b2fd49c2cffa5a6b72f8 |
|
BLAKE2b-256 | 45f9eb3341f608fb12c55845f98d065dde55e58dea0a7c98171eb2e4ffcda24c |
Hashes for blosc2-2.1.1-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7383cd89a41bd22148950e180ca72e3461bf57559e64f57abc5b1cc3e9568089 |
|
MD5 | 24735e8c8fc2ee530d409d0677eef751 |
|
BLAKE2b-256 | 627a10d4b2d9318a99ed84f076f587038b17021b94acbb6123fab087e203d561 |
Hashes for blosc2-2.1.1-cp39-cp39-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 350c892b2a9856bcef3e303f7587f80ea34875e2792f5ce646f0a8b4ce5f8fd4 |
|
MD5 | 71c1c3e882fad61b8b083f3041c51fa3 |
|
BLAKE2b-256 | 9543556b7bc601cb4a1c157c9aa1570a9766f8be903f38801464435e8c9fdea8 |
Hashes for blosc2-2.1.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7bd1a68b5b3c1f101019ac3fb3a78afe9009d59234ea2766a13cfd52a668a7cf |
|
MD5 | 7aaa84424b843b5a0d840d2d27aa6883 |
|
BLAKE2b-256 | 1aaebc35674172a1677eee232eee9ac2ba5b2d890ec61c8f2b8be152d6933ff7 |
Hashes for blosc2-2.1.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 112107148815454f7e3be3234c0e22768c3928984cd5a4121db8c94ea2d18648 |
|
MD5 | 236b86c920c845a839142ded7e34508b |
|
BLAKE2b-256 | b6da3af279146746276c58a55bec60f2a92b3095b489544ceca800bf4d963531 |
Hashes for blosc2-2.1.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a66f0da2bcb35553816a694208409e510c82f589e8f1fdb54ab158c42331f177 |
|
MD5 | 437454a08a370fc44fba52eb6eeb8fa9 |
|
BLAKE2b-256 | d571da72832f1f27e3a81aab35dc6da8992bc89019a168827fe6c95b2d7661c3 |
Hashes for blosc2-2.1.1-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 41d66a1f927ab2967adf9f316d7b3bfe494a9ad9a3145036a4755c1124259154 |
|
MD5 | 6e4dc07d72f1b38e9011f35c7596f604 |
|
BLAKE2b-256 | 7f75cbfb1e51d51d93d619a853077e4a3da26e608f6084bb1ac098b23983ccad |
Hashes for blosc2-2.1.1-cp38-cp38-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4895a5b0ea9522351764c1d8a870c95341447027030e32cb7e5e17ae99e39053 |
|
MD5 | e05aff5f4300b6c459bc92aa3ff5f000 |
|
BLAKE2b-256 | ff64ae8d74cd296062716b05f64535bbe8e009f55a6411014360bdc6a76b60e6 |
Hashes for blosc2-2.1.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e96060365d28db1eb6f40d6972c94d8910b4e38bd418735ddcf8a86005a6e868 |
|
MD5 | 1e83b60072d5845677ac8c9663273f9f |
|
BLAKE2b-256 | 22fae14568bb646dcb19fe7913312a3f22fcaaf1d108810dba430183d3841aac |
Hashes for blosc2-2.1.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 437ecd4bf6a3b0ae2442593d5139e9eed194e6c4b71de910093529584dfc4b87 |
|
MD5 | ef43989c46e2462920da5a6f0c4c1803 |
|
BLAKE2b-256 | f98c0eb647994c7d0ca773fe4dc443e10696cacd0415383f857044f647ec42ce |
Hashes for blosc2-2.1.1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c641ab5e59b59e7de2ae95fa126bf5572f5a7530564765efdaf9ea359d7440e9 |
|
MD5 | fd48f4262605db178e187cb2d83b9320 |
|
BLAKE2b-256 | 5e43435ed57196e79ee245383012808b810a6247ebb94f32364981a9d54eb255 |
Hashes for blosc2-2.1.1-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7e11c61a1e298735efecde4a8c8739636a07ec822af68c85dc572935ea291065 |
|
MD5 | d881f99b055b21f7de5e6cbe09c872e8 |
|
BLAKE2b-256 | 0df239a9bdab8513f9e6190eb4bb524025bec6592c715a09421e3f39e57fdb07 |