Opinionated JSON to CSV converter
Project description
Flatterer. Making JSON flatterer
Introduction
An opinionated JSON to CSV/XLSX converter which tries to make a useful relational output for data analysis.
It aims to be fast and memory efficient.
Install
pip install flatterer
Flatterer requires Python 3.6 or greater. It is written as a python extension in Rust but has binaries (wheels) for linux (x64), macos (x64 and universal) and windows (x64, x86). On other platforms a rust toolchain will need to be installed.
Example JSON
Say you have a JSON data like this named games.json
:
[
{
"id": 1,
"title": "A Game",
"releaseDate": "2015-01-01",
"platforms": [
{"name":"Xbox"},
{"name":"Playstation"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
},
{
"id": 2,
"title": "B Game",
"releaseDate": "2016-01-01",
"platforms": [
{"name":"PC"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
}
]
Running Flatterer
Run the above file with flatterer.
flatterer games.json games_dir
Output Files
By running the above you will get the following files:
tree games_dir
games_dir/
├── csv
│ ├── games.csv
│ └── platforms.csv
├── data_package.json
├── fields.csv
└── ...
Main Table
games.csv
contains:
_link | _link_games | id | rating_code | rating_name | releaseDate | title |
---|---|---|---|---|---|---|
1 | 1 | 1 | E | Everyone | 2015-01-01 | A Game |
2 | 2 | 2 | E | Everyone | 2016-01-01 | B Game |
Special column _link
is generated. _link
is the primary key there unique per game.
Also the rating
sub-object is promoted to this table it has a one-to-one relationship with games
.
Sub-object properties are separated by '_'.
One To Many Table
platforms
is an array so is a one-to-many with games therefore needs its own table:
platforms.csv
contains:
_link | _link_games | name |
---|---|---|
1.platforms.0 | 1 | Xbox |
1.platforms.1 | 1 | Playstation |
2.platforms.0 | 2 | PC |
Link Fields
_link
is the primary key for the platforms
table too. Every table except games
table, contains a _link_games
field to easily join to the main games
table.
If there was a sub-array of platforms
then that would have _link
, _link_games
and _link_platforms
fields.
To generalize this the _link__<table_name>
fields joins to the _link
field of <table_name>
i.e the _link__<table_name>
are the foreign keys refrencing <table_name>._link
.
Fields CSV
fields.csv
contains some metadata about the output tables:
table_name | field_name | field_type | count | field_title |
---|---|---|---|---|
platforms | _link | text | 3 | _link |
platforms | _link_games | text | 3 | _link_games |
platforms | name | text | 3 | name |
games | _link | text | 2 | _link |
games | id | number | 2 | id |
games | rating_code | text | 2 | rating_code |
games | rating_name | text | 2 | rating_name |
games | releaseDate | date | 2 | releaseDate |
games | title | text | 2 | title |
The field_type
column contains a type guess useful for inserting into a database. The field_title
is the column heading in the CSV file or XLSX tab, which is initally the same as the field_name.
After editing this file then you can rerun the transform:
flatterer games.json new_games_dir -f myfields.csv --only-fields
This can be useful for renameing columns, rearranging the field order or if you want to remove some fields the --only-fields
flag will only include the fields in the edited file.
data_package.json
contains metadata in the Tabular Datapackge Spec
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for flatterer-0.10.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e5ac9387b0a2a32a63b281230a0c3d0b5f4c1fe7cc5bf78f497e10b0b0bf20f7 |
|
MD5 | b3e0844db8e4e945e664fdce43409f70 |
|
BLAKE2b-256 | 140d3cb87345c584659c6d9384c56bb16a0e5b4932efa7eae4294d51eb1cf6f5 |
Hashes for flatterer-0.10.1-cp310-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9bfee41c6a2df147e3043304c7da343c37233508acbe5f057ccdff84f833d282 |
|
MD5 | 1793d5f339689f3dbdd3aee6bf02fae8 |
|
BLAKE2b-256 | afb573db9555fc7a56c0f1701843debd6e7d5435e47b2e3e995c33bb31ec21dd |
Hashes for flatterer-0.10.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5bd4dc92072478a6cc5de2884e7b201acfb3da6fe06f47761dadcdab911be44b |
|
MD5 | d5e5f5b93a2b016183d4d11f119f49e0 |
|
BLAKE2b-256 | 0eb34ea2b7bdc12943b68881ba5f368e74b62dd9db46ae9c5036310bcc4a38f4 |
Hashes for flatterer-0.10.1-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e9e872f205ec8713b479b2b64c6ef3ab6082f09fdaadd95bb190e53e264a67b7 |
|
MD5 | e001ab6f4be3137ef6234f1213ba7f5b |
|
BLAKE2b-256 | 14ade567b6b2deb9c9ede54d6f3f2396ad22ae6dc0ad5bec985fe14ee74e9a6f |
Hashes for flatterer-0.10.1-cp310-cp310-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2d1480596ba3a7ff5da9e7a93362b74204a1a6fc4e741763d2b60ef43e52a1f |
|
MD5 | 9f1d433a85a8ceeab4b5da656624e62c |
|
BLAKE2b-256 | 1186878996c27341f94417dcaa09706dc66324cb61611f222eb3121a47dfcc4b |
Hashes for flatterer-0.10.1-cp39-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1acc2a287ae551ebd8ad7a53c71c560dbc3fef05da32ee6968f018465d6d47df |
|
MD5 | a4db35241239109d2bfeb550540bb7e9 |
|
BLAKE2b-256 | 853aa39bcb81d752b908c9e6e2322bcd5501b37df2b10e20e06709d1681d5406 |
Hashes for flatterer-0.10.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0a447f01e47209d4010974245f25680fcb5a0425a00d283ab3d1680c86f87441 |
|
MD5 | 91bb9ee30b1a1b0bc4a460a3e54917a8 |
|
BLAKE2b-256 | 49677093c2ccd0102d46d2b7e50dfbe4f3a65813094db451f671927862848983 |
Hashes for flatterer-0.10.1-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b270d5971241a706518a6c41c7d2a0bc254d14dd9db81a3ee3e61247af6fc25c |
|
MD5 | 9f56755588e63e26e5336b8a2a97913b |
|
BLAKE2b-256 | ccec49396445c20f2429d07f088c418ff04e67a9e3bed2d3f3806fc56bed1994 |
Hashes for flatterer-0.10.1-cp39-cp39-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd564ec1a9bfe16d438c50b9f5016602c4eb4ef9ae79233cb466dee4e9016f2d |
|
MD5 | 3a2efea0f73d537be24f46e66574a5db |
|
BLAKE2b-256 | 636533ba4d807e4e44b6b0ebe26df208320cbd1a8598293b48ec8b1efa54f33d |
Hashes for flatterer-0.10.1-cp38-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cc454f22f0e2f79efd8422cb46db1e4308280d6898de0c101f2b745053eb574a |
|
MD5 | 1e61816976d363c14b172b7d5283d79d |
|
BLAKE2b-256 | 9894c1d31d7b0d6a2caa6bf295a8405fe9d898d94f120d5bccd8190058b02816 |
Hashes for flatterer-0.10.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9c26b75b9c6577f92a3e4a4ac7beb98a5657d783b65c8ee10080519ed81470ef |
|
MD5 | a8ca4bb43daa162cc99e673d83cf1795 |
|
BLAKE2b-256 | 4d6086d9544daf22710755544360e8c4a437d89cde4592210a75400e69289213 |
Hashes for flatterer-0.10.1-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5fe4cec54bafffa03dcfe5b9ad4f00396e766da507a864aaadcf82794af3a113 |
|
MD5 | fe0b6735b85ef40cf4665a966932f1fc |
|
BLAKE2b-256 | 6742c7f825432d43152ad18943c8a1a1ad1f1e8b9dcfb99e1f0b50e0e02d9b31 |
Hashes for flatterer-0.10.1-cp38-cp38-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6fa5b550ddb4bce8bb23a89e64bd2663e323ba8dc50ee4365f2eeb5c77361836 |
|
MD5 | a1e2576db6d29f03dc8eee8029f652d4 |
|
BLAKE2b-256 | ade2a1a0ed15c7e09608b2145583272a1c3497b64ef8db489c046653143ce3a8 |
Hashes for flatterer-0.10.1-cp37-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e0b48a8275ace174d5ebd6b1b2becca1ec7a570b440ee2c7f066e39ff723f49 |
|
MD5 | 0d8f7d95fea5fa951a2639641f95ac58 |
|
BLAKE2b-256 | 183356e9daec165de760ec8ca271aa46992fcd5c67d62102ad6bc7e5b97ee283 |
Hashes for flatterer-0.10.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fb19d66f3d3ce87c08c77dbe37a687c34ea35c0076bea1b6e80c87d6256539dc |
|
MD5 | 2ab3d5160f0d881414b8fa68a3df6029 |
|
BLAKE2b-256 | 1eba8f6e879383d48b5ffc3f187dcc9a58c3777243a14e0784ce02ea4aae172c |
Hashes for flatterer-0.10.1-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b2a60deec9825da472b296b69982c642b77188f6a7ba7e1f0149598c392e4bed |
|
MD5 | db88b5339f912325f6f697a6995b31c0 |
|
BLAKE2b-256 | f01aff93433015691d19ca7aaccfb7ea8f2116e2f32c036216b6f50823af8ac0 |
Hashes for flatterer-0.10.1-cp37-cp37m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fadbd0c1c4044d9cafa3141f903211ec52bb80d72c81f6d151567384b8e31415 |
|
MD5 | ae8f6f68de89ef53f1a2e7639d99ce2f |
|
BLAKE2b-256 | 7e66813286f94b99ce4933d313b58e9626469dc4115a16afd8d37784f8b59f53 |
Hashes for flatterer-0.10.1-cp36-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 89234f91fb45d7620c97d84652e589b2e41ec88c0248b2dd6be4e721e2a4a5f3 |
|
MD5 | 6ebbb0cfd340e23ca253b711334cdc15 |
|
BLAKE2b-256 | 94ede1e91dc26a579e3abcd6ffe1132ef0082fd5afa5280e7c6ac6dbadb7b221 |
Hashes for flatterer-0.10.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 769cfebe3116a080415d905a9bc4a51f997b3e892d06de2eca01c5e3010b7006 |
|
MD5 | 07176ad63eb52dfdc6d9d08222c9dce9 |
|
BLAKE2b-256 | 1d2a79834e0d7088ec9cbf21a768bdafe282c6146df976a08e6f2cedc6e6b047 |
Hashes for flatterer-0.10.1-cp36-cp36m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fd4f97a610c18d6b3930e4d90755528c4b57f0adcafcae6dab9872efb4b123cd |
|
MD5 | 9622d0df89d85a64208e0c5045526191 |
|
BLAKE2b-256 | baf21ba8a9aec2912f0569477e5e74de3fe06b334610c4bd024c7ca7fd0c045d |
Hashes for flatterer-0.10.1-cp36-cp36m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8fe6b6df051e7bc18efa4417dfaabe0caefb6b221c9b47b3d37325d8a34b264 |
|
MD5 | 55430b5bf9c2a5268eea9ea076a73131 |
|
BLAKE2b-256 | 65e0afc566b8c5c1736c6d669ad6afd8f80faa862b7709680a7da1ae319b4f52 |