Opinionated JSON to CSV converter
Project description
Flatterer. Making JSON flatterer
Introduction
An opinionated JSON to CSV/XLSX converter which tries to make a useful relational output for data analysis.
It aims to be fast and memory efficient.
Install
pip install flatterer
Flatterer requires Python 3.6 or greater. It is written as a python extension in Rust but has binaries (wheels) for linux (x64), macos (x64 and universal) and windows (x64, x86). On other platforms a rust toolchain will need to be installed.
Example JSON
Say you have a JSON data like this named games.json
:
[
{
"id": 1,
"title": "A Game",
"releaseDate": "2015-01-01",
"platforms": [
{"name":"Xbox"},
{"name":"Playstation"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
},
{
"id": 2,
"title": "B Game",
"releaseDate": "2016-01-01",
"platforms": [
{"name":"PC"}
],
"rating": {
"code": "E",
"name": "Everyone"
}
}
]
Running Flatterer
Run the above file with flatterer.
flatterer games.json games_dir
Output Files
By running the above you will get the following files:
tree games_dir
games_dir/
├── csv
│ ├── games.csv
│ └── platforms.csv
├── data_package.json
├── fields.csv
└── ...
Main Table
games.csv
contains:
_link | _link_games | id | rating_code | rating_name | releaseDate | title |
---|---|---|---|---|---|---|
1 | 1 | 1 | E | Everyone | 2015-01-01 | A Game |
2 | 2 | 2 | E | Everyone | 2016-01-01 | B Game |
Special columns _link
and _link_games
are generated. _link
is the primary key there unique per game.
Also the rating
sub-object is promoted to this table it has a one-to-one relationship with games
.
Sub-object properties are separated by '_'.
One To Many Table
platforms
is an array so is a one-to-many with games therefore needs its own table:
platforms.csv
contains:
_link | _link_games | name |
---|---|---|
1.platforms.0 | 1 | Xbox |
1.platforms.1 | 1 | Playstation |
2.platforms.0 | 2 | PC |
Link Fields
_link
is the primary key for the platforms
table too. Every table contains a _link_games
field to easily join to the main games
table.
If there was a sub-array of platforms
then that would have _link
, _link_games
and _link_platforms
fields.
To generalize this the _link__<table_name>
fields joins to the _link
field of <table_name>
i.e the _link__<table_name>
are the foreign keys refrencing <table_name>._link
.
Fields CSV
fields.csv
contains some metadata about the output tables:
table_name | field_name | field_type | count | field_title |
---|---|---|---|---|
platforms | _link | text | 3 | _link |
platforms | _link_games | text | 3 | _link_games |
platforms | name | text | 3 | name |
games | _link | text | 2 | _link |
games | _link_games | text | 2 | _link_games |
games | id | number | 2 | id |
games | rating_code | text | 2 | rating_code |
games | rating_name | text | 2 | rating_name |
games | releaseDate | date | 2 | releaseDate |
games | title | text | 2 | title |
The field_type
column contains a type guess useful for inserting into a database. The field_title
is the column heading in the CSV file or XLSX tab, which is initally the same as the field_name.
After editing this file then you can rerun the transform:
flatterer games.json new_games_dir -f myfields.csv --only-fields
This can be useful for renameing columns, rearranging the field order or if you want to remove some fields the --only-fields
flag will only include the fields in the edited file.
data_package.json
contains metadata in the Tabular Datapackge Spec
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for flatterer-0.9.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b6293001255b340bd4811e0d3b45dfd5f5d47bc088bc63119772b04003e83a68 |
|
MD5 | d45715c1b26296b21d3922c412729394 |
|
BLAKE2b-256 | 1654373b8f10e40bc82d4ac32c6e05dbe733f385578699be9fa44c4db5e6089b |
Hashes for flatterer-0.9.0-cp310-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 375bfb536d6f14177c1b98cf03d7dfa226b61d44c567d987032eced68ebd2397 |
|
MD5 | 8b243d23d8baeca14b7d9eda13d4a077 |
|
BLAKE2b-256 | faf5fad137219e86e4095711718a3fd472b31ca4eb737d2392e0bdb22f5b5286 |
Hashes for flatterer-0.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c61732500187b1a0a1b84cda25197875b6c55efafdb53e892bebbf5021f2b949 |
|
MD5 | a5fa33f70b022dd5c3ecc6fd1771fd43 |
|
BLAKE2b-256 | 46052eed83c50b12c0fe0dec392a9d08182c2b778d3c6dd7819bf35ea8db16ba |
Hashes for flatterer-0.9.0-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e47173ad7a33645f3469a419c4a68aeb35beec259d746bcce622f2e5496686c8 |
|
MD5 | 3671cd94454fbc57d341fc59b9eaf3f3 |
|
BLAKE2b-256 | 6e77b4114c3ff107bdbee6faf9a04a056b1d5f0d9b506fb3c75e4e12cb770a03 |
Hashes for flatterer-0.9.0-cp310-cp310-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3189a36ac92edf462b3f7a227c212c12a53318882b6533b0ee8428de4c2c5aa7 |
|
MD5 | 0a081c22d1e9b94a4603855e4fb6bf83 |
|
BLAKE2b-256 | d92811e8aeb417b68da3f07e0f08c53cb023a91bc3ae027ada190557321bc888 |
Hashes for flatterer-0.9.0-cp39-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 626e356c211e5634af5c3cfa0a29c61b380acb48b86528c980f7dbc1dca6989c |
|
MD5 | be0011dd3763033f1940ba4bdce3749f |
|
BLAKE2b-256 | c3ad1bdbcea846f2df5d995e2733661e22b2b9b0eb9be1cc9fca95907586fb80 |
Hashes for flatterer-0.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 00ca0f4e2eac696c97575ec075362c284eb22903347289eb84db9e6b59a30bda |
|
MD5 | ef3f7be51d53fd4df53fedeb373864f1 |
|
BLAKE2b-256 | a10d30ca450e23142d083ad1ba9c5aed5c534b9a4b28e3a695cc6db476333135 |
Hashes for flatterer-0.9.0-cp39-cp39-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d1ed3f9a04881f2fd42ca5d6fa03ead88422997660633fd9ff0f5a10877a501f |
|
MD5 | afdb39e5f914c2e9bed56db1d02d4272 |
|
BLAKE2b-256 | 7eb689ed3bfac440c418bf4a66e27a2740f45e2ae966d58107ebaff16bd01b69 |
Hashes for flatterer-0.9.0-cp39-cp39-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1293bab90b240a25e7a9be2353225bc5fd40f4c24da411ebd076457107d01bb9 |
|
MD5 | 1e4976bffafe02dbb55af24916f004e8 |
|
BLAKE2b-256 | 8bc6bd398979ac9e5a59b51d23fbaca6e4a72967ea8c33544613b3b2391a7913 |
Hashes for flatterer-0.9.0-cp38-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c698d7a9765ae0304ff816440ae0d8f3d5a1219ee5bd9b280a75a175b8e8d323 |
|
MD5 | 95bb8664e38186b746ca32b1740943a0 |
|
BLAKE2b-256 | 70aa06e48c6312bf2f0c44c61c673c7c7170478d5d3a61ef86cddea168aa63e3 |
Hashes for flatterer-0.9.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f99dc62a071083b174b6458a630fc3f9496e76d55b8cbe384cbaa97d9edd04a6 |
|
MD5 | f4ef5e96a715efab95e7cbd86cc0d71a |
|
BLAKE2b-256 | 15ea07f592a97fd5f25f7d0a072ae81bca9db0986f68f52472439955f98e755d |
Hashes for flatterer-0.9.0-cp38-cp38-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 716113011786fc59e3b04fe128ead64888482e354543076bdec61fcc9b8bd3ba |
|
MD5 | e42fb2a3d24848d769b4a550224033cc |
|
BLAKE2b-256 | a436a03c5fa216152b30b2e2c227c1ccd5a866474d00be85143869cad6597e0a |
Hashes for flatterer-0.9.0-cp38-cp38-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | df771ffebd76c3c446f3f49a4822c03e95e7791894cdbe40036d7b068f9f4a28 |
|
MD5 | dc86d1f0067b92fe564ce0104572b07c |
|
BLAKE2b-256 | ffc4123e7f875844248c5783cee5bae9a47b75e82c5f3a859a725bfd2a2d6e5f |
Hashes for flatterer-0.9.0-cp37-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e8df03e89e3f6ae424d7737e5c30540f9b84e3275dc4ffcb7ebdccd37f6230dc |
|
MD5 | 86925bc1e8ae4d1139f12042aada7098 |
|
BLAKE2b-256 | 2f6d697f1f2741decc5bcd5eed16a96a76df0d2383901a60178d19baeee0236d |
Hashes for flatterer-0.9.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8b068dfbbbe870db0956d7005a074a267fff10d754cfdc80d83e816816c5509 |
|
MD5 | 875a5a8ecc3f55912f0aa5d410c27b00 |
|
BLAKE2b-256 | d7513ec6a7c610d8522643bdd357d742631672f0d6049a50fc16ca63885479b1 |
Hashes for flatterer-0.9.0-cp37-cp37m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2143c37c5b1287c77a480e693dc6ea3211cb13294da1c259adde279d0ab4ca0f |
|
MD5 | 5765189bce837683b47e3a0c3e3ccdec |
|
BLAKE2b-256 | f15762f4ba2084eba16bd4ea5d5013056adccd7643ac5c6967d2301d797736ba |
Hashes for flatterer-0.9.0-cp37-cp37m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f603d47edb415ae8613f71e7bc2c2d21743b247bdfe6d51c7fe91ff6a70c8060 |
|
MD5 | 9aed84f33674f9fda46209851e3b7889 |
|
BLAKE2b-256 | 87bd9b8dec31b2bad57ecf824a1cde352e0624206a126e5ba96bf56f7b1e0287 |
Hashes for flatterer-0.9.0-cp36-none-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 11918c2d6513e507218ec817076825fdb92ef1be00e8eb2997aa9aa817a6d976 |
|
MD5 | 082dcade433a5a2922cc2bff84a6a958 |
|
BLAKE2b-256 | 86c1eab776eb3ab8eae064c0b6c69c706d7d14ac70aad0e5af20b9ce05e1fc95 |
Hashes for flatterer-0.9.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 913dd30a889db588cf3e976fcfd9acf37916854fd66c4e8f5dbad4e9a636fa7f |
|
MD5 | b0a7f40d548aa4851f28db56adb64b0f |
|
BLAKE2b-256 | cd8de6478effca2867fa13031a44e1541f1d972241e5911436887227dfdb2501 |
Hashes for flatterer-0.9.0-cp36-cp36m-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ead2bad4e4c0e55fa67bc5027770cd2ddc18780c673786ec1a1213b7eb12589b |
|
MD5 | 7e6bd75f13f24e1c18b8607ace95f211 |
|
BLAKE2b-256 | 3971a881bf8762719bf0064e870f2ba32e74f5006e01d9ebb2fd38a5d0026fe7 |
Hashes for flatterer-0.9.0-cp36-cp36m-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6b45b52ecac474127249277943346652ecdbfb21ddc98bcc480f5bfc402c2983 |
|
MD5 | 2e3118fc82eecb05624c8764f96bbd09 |
|
BLAKE2b-256 | 181743c571b0a03d448ed16ba6a5ebb3cf1e501cc13951ca1e7612388c6951aa |