Provides utility functions for accessing data repository for Project Pythia examples/notebooks
Project description
CI | |
---|---|
Docs | |
Package | |
License |
pythia-datasets
Data repository for Project Pythia examples/notebooks
Sample data sets
These files are used as sample data in Pythia Project examples/notebooks and are downloaded by pythia_datasets
package:
NARR_19930313_0000.nc
enso_data.csv
jan-17-co-asos.txt.xz
Adding new datasets
To add a new dataset file, please follow these steps:
- Add the dataset file to the
data/
directory - From the command line, run
python make_registry.py
script to update the registry file residing inpythia_datasets/registry.txt
- Commit and push your changes to GitHub
Using datasets in notebooks and/or scripts
-
Ensure the
pythia_datasets
package is installed in your environmentpython -m pip install pythia-datasets # or python -m pip install git+https://github.com/ProjectPythia/pythia-datasets
-
Import
DATASETS
and inspect the registry to find out which datasets are availableIn [1]: from pythia_datasets import DATASETS In [2]: DATASETS.registry_files Out[2]: ['jan-17-co-asos.txt.xz', 'NARR_19930313_0000.nc']
-
To fetch a data file of interest, use the
.fetch
method and provide the filename of the data file. This will- download and cache the file if it doesn't exist already.
- retrieve and return the local path
In [4]: filepath = DATASETS.fetch('jan-17-co-asos.txt.xz') In [5]: filepath Out[5]: '/Users/abanihi/Library/Caches/pythia-datasets/jan-17-co-asos.txt.xz'
-
Once you have access to the local filepath, you can then use it to load your dataset into pandas or xarray or your package of choice:
In [6]: df = pd.read_csv(filepath)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pythia-datasets-2021.6.11.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35f39402ae9fd7a27118313910c30217e5b8c703287860f148d340d252adf1d7 |
|
MD5 | cffad112dd69a72c448150cc919f7ce4 |
|
BLAKE2b-256 | b18ae4f6aa1b6b6e03d6d2da80decf2591983835e997c28189328b9019f15047 |
Hashes for pythia_datasets-2021.6.11-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e934de8d6182680598386da848f35d0abe3d6348fb2922bd0d42b7e27a63c35c |
|
MD5 | 35e99ffa9df90ab1bd03629a6d22058c |
|
BLAKE2b-256 | ba62d93caabc940b4ad335dff8e8306a0f226bf3c9e8107bf546d8ce08269a04 |