Opinionated JSON to CSV converter
Project description
Flatterer. Making JSON flatterer
Introduction
An opinionated JSON to CSV/XLSX converter which tries to make a useful relational output for data analysis.
It aims to be fast and memory efficient.
Install
pip install flatterer
Flatterer requires Python 3.6 or greater. It is written as a python extension in Rust but has binaries (wheels) for linux (x64), macos (x64 and universal) and windows (x64, x86). On other platforms a rust toolchain will need to be installed.
Example JSON
Say you have a JSON data like this named games.json
:
[
{
"id": 1,
"title": "A Game",
"releaseDate": "2015-01-01",
"platforms": [
{"name":"Xbox"},
{"name":"Playstation"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
},
{
"id": 2,
"title": "B Game",
"releaseDate": "2016-01-01",
"platforms": [
{"name":"PC"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
}
]
Running Flatterer
Run the above file with flatterer.
flatterer games.json games_dir
Output Files
By running the above you will get the following files:
tree games_dir
games_dir/
├── csv
│ ├── games.csv
│ └── platforms.csv
├── data_package.json
├── fields.csv
└── ...
Main Table
games.csv
contains:
_link | _link_games | id | rating_code | rating_name | releaseDate | title |
---|---|---|---|---|---|---|
1 | 1 | 1 | E | Everyone | 2015-01-01 | A Game |
2 | 2 | 2 | E | Everyone | 2016-01-01 | B Game |
Special column _link
is generated. _link
is the primary key there unique per game.
Also the rating
sub-object is promoted to this table it has a one-to-one relationship with games
.
Sub-object properties are separated by '_'.
One To Many Table
platforms
is an array so is a one-to-many with games therefore needs its own table:
platforms.csv
contains:
_link | _link_games | name |
---|---|---|
1.platforms.0 | 1 | Xbox |
1.platforms.1 | 1 | Playstation |
2.platforms.0 | 2 | PC |
Link Fields
_link
is the primary key for the platforms
table too. Every table except games
table, contains a _link_games
field to easily join to the main games
table.
If there was a sub-array of platforms
then that would have _link
, _link_games
and _link_platforms
fields.
To generalize this the _link__<table_name>
fields joins to the _link
field of <table_name>
i.e the _link__<table_name>
are the foreign keys refrencing <table_name>._link
.
Fields CSV
fields.csv
contains some metadata about the output tables:
table_name | field_name | field_type | count | field_title |
---|---|---|---|---|
platforms | _link | text | 3 | _link |
platforms | _link_games | text | 3 | _link_games |
platforms | name | text | 3 | name |
games | _link | text | 2 | _link |
games | id | number | 2 | id |
games | rating_code | text | 2 | rating_code |
games | rating_name | text | 2 | rating_name |
games | releaseDate | date | 2 | releaseDate |
games | title | text | 2 | title |
The field_type
column contains a type guess useful for inserting into a database. The field_title
is the column heading in the CSV file or XLSX tab, which is initally the same as the field_name.
After editing this file then you can rerun the transform:
flatterer games.json new_games_dir -f myfields.csv --only-fields
This can be useful for renameing columns, rearranging the field order or if you want to remove some fields the --only-fields
flag will only include the fields in the edited file.
data_package.json
contains metadata in the Tabular Datapackge Spec
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for flatterer-0.11.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d55b04793269787afb9783ba0c8ba3563e4c908b55abe86290aa93e0deda53f7 |
|
MD5 | c507494e714cd023ed2631bfaedd7ddc |
|
BLAKE2b-256 | 619eb94af524552d12218d054278b83231acbf341d0eee66beedc4d225034771 |
Hashes for flatterer-0.11.0-cp310-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6f35ce40775f2fe099b9c906b87f5f285db4d245f49f3f58ae07fd44c4af09bb |
|
MD5 | 6d218799875093f4061ca63e74d09081 |
|
BLAKE2b-256 | b26adf6660ab2573426c0c9b7a2c0c0b45b66b3cd47847dda3360ca3c0e8fd0e |
Hashes for flatterer-0.11.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d076926ead4fe4ad2b00fe20412cb9d0441f1730b0509f8350b8cb8b69829664 |
|
MD5 | d8af81d8c4573b9bc59c38daaab05801 |
|
BLAKE2b-256 | a1c967a5129cecc29f3aa4e5c357309fbc9f0430062f616bf1febcdc00b61927 |
Hashes for flatterer-0.11.0-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 740562c0c2da5e06b503c92983898cb8ed93a72fb1a7ac0816acf48fef9ed931 |
|
MD5 | 3191fe07a4ddb9877a8b4f6c4ce57045 |
|
BLAKE2b-256 | 49a8b862f277a25b913402bfcfb173d5f07570e173d345ecff79f2036d7dab22 |
Hashes for flatterer-0.11.0-cp310-cp310-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 377290877bb4e1a040415bb6e98452186c8ebe2378f13fb16852df5d56a0dfe2 |
|
MD5 | 3397af919bd3c15c85f47c7c896bcdfe |
|
BLAKE2b-256 | 4942aed5f06f808b8e8850aa6f3bb0cc280c23ae8ce70539fce92c6cc6eebb24 |
Hashes for flatterer-0.11.0-cp39-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd97b708f355dc454ba685fd3332ba45bf4535e9a2c4c6351c75e3b16cdbdf13 |
|
MD5 | b9a0b189dcd47cc76102a6619fc9f93e |
|
BLAKE2b-256 | acd888cb6a855e25b3b8c8f24e2a90625cecbeafd92e375a81e265ba643983a3 |
Hashes for flatterer-0.11.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 28467548f79fb381e9c0c9e799fc0415eecfc1696e59d22da6ecdd5870db82b3 |
|
MD5 | 03f94684d2e56d4a88dcb55f85cb8dab |
|
BLAKE2b-256 | 2b5e720ba2a9291dffbc874418b8dc343b70603522916c252c096b89ecbec5e5 |
Hashes for flatterer-0.11.0-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3306e0cc1bd88a769e95e4035e0587b05cbd5ecadfdedba3ce910c402cedec34 |
|
MD5 | c474bf75b044db77276245df613764d6 |
|
BLAKE2b-256 | cb945354542a84794f8926d4762cd3fb22a283e441728285c40c5fbec26680e0 |
Hashes for flatterer-0.11.0-cp39-cp39-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d41a29e3bf23f1895164245ee62222ecba7aab2ecf51b8b72777cac956ea0b73 |
|
MD5 | 45e2214ec86ec6d386ac8ab569b12065 |
|
BLAKE2b-256 | 81f7fc955ebb11b99253cd95c278ef3b38de78ec4fab8be0b7ae1b0900288c82 |
Hashes for flatterer-0.11.0-cp38-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 119b8d497cf18f4c73e56111c8fd60788134e9525c98994debeb6264b632d21d |
|
MD5 | 771719eaef3629f7129cc41be96d3d13 |
|
BLAKE2b-256 | 4cce24c9182d06055b19079551094e581118b7c26c37fac5ac08103e1d2853f2 |
Hashes for flatterer-0.11.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 52d772c34bd8878779af516422bbcb5d3f3b606dbab2b4f02cce96187558f13c |
|
MD5 | 24365b2b4f01315722742501e6d9d201 |
|
BLAKE2b-256 | 14e6f33ff2da2e626429f61ac66e4f9f41a920b350ecd2e0f31c88d9f2615a5b |
Hashes for flatterer-0.11.0-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1e572ada1afa37f7bb9a373493b7ae3a93afdec5fa24c0bcce5e0cfd78e5ebaa |
|
MD5 | 3608f36439af4b9506b88355a72c305c |
|
BLAKE2b-256 | 3bdea638235176a4429a4ededf60555e49a6f50b5b7e8317b8edeefc3126c75c |
Hashes for flatterer-0.11.0-cp38-cp38-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 072c6a10684585656f07e0f2bf740bfd3ba6e59278eba20776c89d733679f746 |
|
MD5 | 21b5e79394e394b2eaab018643efe7f6 |
|
BLAKE2b-256 | dce8df75f60f82b65cd917b2a00540cffc7350df7b7337baf2b2ce6483c80850 |
Hashes for flatterer-0.11.0-cp37-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6d9ece8dadf406207b14b8b3ccb81bb29a8e613dbd52d3770c018bc81f4f0dcf |
|
MD5 | 38c3010d66d5079e4137e654b23c41a5 |
|
BLAKE2b-256 | b08cfdf344959744471db4d032cbfad83a69a4fc5b6980621e7dd1a4b39d02c5 |
Hashes for flatterer-0.11.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6dfa7d11b374752398c2518cb2c78bc3a330313853af0ffbb7b67f7af90757fd |
|
MD5 | d50fbbc9e8889f2a8a0c20d3dded0d71 |
|
BLAKE2b-256 | 9ae3f3a9c012ce8f0167a4a09d27078099ed3d5b64e53f686e97cfb036422658 |
Hashes for flatterer-0.11.0-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cc2b1c2ffaf615fa179c2f70620ca8c3cbcb361d8a64f63cf4e144f197e4cc6f |
|
MD5 | ead6f0a48d0f4b4ff8acf33b12800474 |
|
BLAKE2b-256 | eac67223551a099adf9c801d964cdb7419fbe11d1af7887d678781f5efac05a4 |
Hashes for flatterer-0.11.0-cp37-cp37m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7b557eaeec0246d407d211777eb88667b9ed00d12c9dd735e5a6be74d1a1965f |
|
MD5 | 159512df21b6ab0aca53e917d4efb7b3 |
|
BLAKE2b-256 | f068f8fa4d6ce9754f560d7ed21a6582210939183237afbf5b921e7ee57efe36 |
Hashes for flatterer-0.11.0-cp36-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ff9ec68d36db63931b8e5e08109b60469b3ced9dd8358140259eaaa7d88c2019 |
|
MD5 | dd2358e18a89126b651a2da63c9088b3 |
|
BLAKE2b-256 | d27ef42d301faca886a6978ee8d4b2b2b85d08ff17a7ab61c314ec5baba71cca |
Hashes for flatterer-0.11.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1eee3e8cfc7b69d83bc5d822807804e64936366d5d90dfb3e452bdd9b2d1a22d |
|
MD5 | 871fa488d5d742be35fc32e0bb048d6e |
|
BLAKE2b-256 | a36f79f82fe6bbde3996461400efaa4c129b8db2739b00b512336478e8b9cf25 |
Hashes for flatterer-0.11.0-cp36-cp36m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 55b8e33350fb53028666d17f442c7ed596f4c8e8540cd89fe2570e59a750480a |
|
MD5 | 88d36e482bb630590110e87f86d823f8 |
|
BLAKE2b-256 | b4e90194da42d9688a68ccb7e8fba7f27bb675789b08ddfed2b485ad1f11d9ce |
Hashes for flatterer-0.11.0-cp36-cp36m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a503c1cacf84a275555e7c872387687a857bd1c469513c818cf5cd0323c13ea4 |
|
MD5 | ee25ff929c666ba8bcbd4d164eb6913f |
|
BLAKE2b-256 | 1b0847fa4be31d4ecc1c60f7e59e8fb9ed8cc5dfe858eaec27eb6d15c8200531 |