Provides utility functions for accessing data repository for Project Pythia examples/notebooks
Project description
CI | |
---|---|
Docs | |
Package | |
License |
pythia-datasets
Data repository for Project Pythia examples/notebooks
Sample data sets
These files are used as sample data in Pythia Project examples/notebooks and are downloaded by pythia_datasets
package:
NARR_19930313_0000.nc
enso_data.csv
jan-17-co-asos.txt.xz
CESM2_sst_data.nc
CESM2_grid_variables.nc
Adding new datasets
To add a new dataset file, please follow these steps:
- Add the dataset file to the
data/
directory - From the command line, run
python make_registry.py
script to update the registry file residing inpythia_datasets/registry.txt
- Commit and push your changes to GitHub
Using datasets in notebooks and/or scripts
-
Ensure the
pythia_datasets
package is installed in your environmentpython -m pip install pythia-datasets # or python -m pip install git+https://github.com/ProjectPythia/pythia-datasets
-
Import
DATASETS
and inspect the registry to find out which datasets are availableIn [1]: from pythia_datasets import DATASETS In [2]: DATASETS.registry_files Out[2]: ['jan-17-co-asos.txt.xz', 'NARR_19930313_0000.nc']
-
To fetch a data file of interest, use the
.fetch
method and provide the filename of the data file. This will- download and cache the file if it doesn't exist already.
- retrieve and return the local path
In [4]: filepath = DATASETS.fetch('jan-17-co-asos.txt.xz') In [5]: filepath Out[5]: '/Users/abanihi/Library/Caches/pythia-datasets/jan-17-co-asos.txt.xz'
-
Once you have access to the local filepath, you can then use it to load your dataset into pandas or xarray or your package of choice:
In [6]: df = pd.read_csv(filepath)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pythia-datasets-2021.9.13.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 14a4ac096c25ef4af7b2b1535552119de89aa20e67224745b5dbfb548cf87a61 |
|
MD5 | aa830c9f20b6bde26153622ac66c1b11 |
|
BLAKE2b-256 | 611fe89b00033d0eabc49da8ec042c3c5470d9b55c292ab3fb751feb57712d92 |
Hashes for pythia_datasets-2021.9.13-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f846db484a44cb56e987490796ddd4fb570307efffff740055f58ac23c2e7287 |
|
MD5 | f75bb5f89287dc2df05964a55a3decc8 |
|
BLAKE2b-256 | d6b56b44abb223521fc5a4a222e2b01dbda9cc98f591b5652351f93934783865 |