TileDB Cloud platform Python client
Project description
TileDB-Cloud-Py
This repository contains the Python client for the TileDB Cloud Service.
Quick Links
Installation
Quickstart
See Quickstart section of the docs.
API Documentation
See https://tiledb-inc.github.io/TileDB-Cloud-Py/
Contribute to documentation
Documentation uses Quarto style documentation.
- Install Quarto for your OS
- Install documentation dependencies:
pip install .[docs] - Update the docs/_quarto.yaml file accordingly (read the quartodoc documentation on how to do that)
- Build the docs:
quartodoc build --config docs/_quarto.yaml - Preview the generated docs:
quarto preview docs/ - Create a relevant PR
Testing
- Selection:
- All tests:
pytest - All tests with verbosity:
pytest -vs - Tests with
taskgraphsin their names:pytest -vsk taskgraphs
- All tests:
- Credentials:
- These tests run on TileDB Cloud using your current environment variable
TILEDB_REST_TOKEN-- you will need a valid API token for the tests to pass - For continuous integration, the token is configured for the
unittestuser and all tests should pass - For interactive use, if your
TILEDB_REST_TOKENpoints to your own account, most tests will run, except for those that explicitly check against contents of theunittestaccount which are skipped
- These tests run on TileDB Cloud using your current environment variable
Ingestion testing
This package contains modules and functions that will be run in the TileDB Cloud as UDFs. Local ingestor changes can be tested in the cloud by using a feature of cloudpickle. Below is a runnable example. TILEDB_NAMESPACE is your TileDB namespace ("TileDB-Inc", for example). TILEDB_ACCESS_CREDENTIAL_NAME is the name of the stored credentials for accessing AWS_BUCKET. IMAGE_FILE_KEY is the key for an object in that bucket and OUTPUT_GROUP_KEY is the key to be used for the group that the ingest() UDF will create in AWS_BUCKET.
import cloudpickle
import tiledb.cloud.bioimg
cloudpickle.register_pickle_by_value(tiledb.cloud.bioimg)
tiledb.cloud.bioimg.ingest(
source="s3://AWS_BUCKET/IMAGE_FILE_KEY",
output="s3://AWS_BUCKET/OUTPUT_GROUP_KEY",
config=None,
namespace="TILEDB_NAMESPACE",
acn="TILEDB_ACCESS_CREDENTIAL_NAME",
ingest_resources={"cpu": "8", "memory": "32Gi"},
)
In this case tiledb.cloud.bioimg.ingest() uses cloudpickle to send a local function to TileDB Cloud, and cloudpickle.register_pickle_by_value(tiledb.cloud.bioimg) directs cloudpickle to bring the currently imported tiledb.cloud.bioimg module along with the function. Your local version of the module will be used instead of the version currently deployed in TileDB Cloud.
Note: your local changes to the Cloud-Py package will need to be installed in order for cloudpickle to serialize them, as cloudpickle needs to find them at runtime.
Releasing
Releasing is entirely automated. Releases made on GitHub using tags that start with "v", like "v0.12.28", trigger sdist and wheel builds and upload of those distributions to the Supercacher Package Index.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tiledb_cloud-0.12.29-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 7d7e3166a7c898e9157051a626f7fa8a8ba9c542e1098acb76d65d5e7c14061d |
|
| MD5 | 90b3265cf284d09f95b85dc6524402ff |
|
| BLAKE2b-256 | 59d7d5240e316e6722eb978443eee7a9333bf9438e7f11ed59018aeaa306a26b |
