Provides utility functions for accessing data repository for Project Pythia examples/notebooks
Project description
CI | |
---|---|
Docs | |
Package | |
License |
pythia-datasets
Data repository for Project Pythia examples/notebooks
Sample data sets
These files are used as sample data in Pythia Project examples/notebooks and are downloaded by pythia_datasets
package:
NARR_19930313_0000.nc
enso_data.csv
jan-17-co-asos.txt.xz
CESM2_sst_data.nc
CESM2_grid_variables.nc
Adding new datasets
To add a new dataset file, please follow these steps:
- Add the dataset file to the
data/
directory - From the command line, run
python make_registry.py
script to update the registry file residing inpythia_datasets/registry.txt
- Commit and push your changes to GitHub
Using datasets in notebooks and/or scripts
-
Ensure the
pythia_datasets
package is installed in your environmentpython -m pip install pythia-datasets # or python -m pip install git+https://github.com/ProjectPythia/pythia-datasets
-
Import
DATASETS
and inspect the registry to find out which datasets are availableIn [1]: from pythia_datasets import DATASETS In [2]: DATASETS.registry_files Out[2]: ['jan-17-co-asos.txt.xz', 'NARR_19930313_0000.nc']
-
To fetch a data file of interest, use the
.fetch
method and provide the filename of the data file. This will- download and cache the file if it doesn't exist already.
- retrieve and return the local path
In [4]: filepath = DATASETS.fetch('jan-17-co-asos.txt.xz') In [5]: filepath Out[5]: '/Users/abanihi/Library/Caches/pythia-datasets/jan-17-co-asos.txt.xz'
-
Once you have access to the local filepath, you can then use it to load your dataset into pandas or xarray or your package of choice:
In [6]: df = pd.read_csv(filepath)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pythia-datasets-2021.9.21.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19fcb86b0db71aa3251dcab528b51bf66639e2dc3c8654d6fc80b8a3379ac650 |
|
MD5 | 28aa736e264964a7699031ce22b102d9 |
|
BLAKE2b-256 | 0b45fd687d08636413a2b612b56ce67e83fe1724158a962199d9fc6e45f04d5b |
Hashes for pythia_datasets-2021.9.21-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 179dbe46538ad2310fd11075472807c425ffa956db6b90957d52211e22441f5b |
|
MD5 | 2c6f7fc3afb67dc36253c5b152d26516 |
|
BLAKE2b-256 | e540fc8ffc2c2fb3bbb0dec75bec4a5fb5336f035ceafac5397400f9c2b12dab |