Support JSON as input (and output?) format

[mechatroner]

One of the main problems is supporting JOIN, when we have a query like select * JOIN other_file.json on a['key'] = b['key']['subkey'], what should the output records be? One option would be to output [a, b] which would be valid json too. We would also have to create an alternative join map generation that would have to be eval-based, currently we convert join variables to indexes to generate the join map, but with JSON it is not possible.

[mechatroner]

Actually, if we allow JSON as output, the exact output record structure shouldn't be viewed as a huge decision point, since with JSON output users can easily create arbitrary output structures, example:
SELECT json.dumps([a, b, {'foo': a['bar'], 'foobar': {'b': b}}]) join other_file.json on a['key'] == b['key']

Much bigger question is what should be the output records structure in case of json input and csv output (i.e. query + format conversion)

[JoeCodeswell]

Hi @mechatroner,

How hard is just doing json.dumps for a very simple object? Then just say that's the limitation for now?

Thanks for the good work.

Love and peace,
Joe

[mechatroner]

Hi @JoeCodeswell, Thank you for the feedback!
Yeah, json.dumps will work - I just verified it, but you would need to add import json in ~/.rbql_init_source.py file. Or you can also use JSON.stringify in JS version out of the box. It is also better to set the output format as tsv instead of csv to avoid double quoting.
So output JSON is already supported to some extent.

[JoeCodeswell]

Thanks for letting me know, Dmitry. Love and peace, Joe

…

On Tue, Feb 13, 2024 at 5:57 PM Dmitry Ignatovich ***@***.***> wrote: Hi @JoeCodeswell <https://github.com/JoeCodeswell>, Thank you for the feedback! Yeah, json.dumps will work - I just verified it, but you would need to add import json in ~/.rbql_init_source.py file. Or you can also use JSON.stringify in JS version out of the box. It is also better to set the output format as tsv instead of csv to avoid double quoting. So output JSON is already supported to some extent. — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYLZJIL52ZCATOY3YPIFMLYTQKYFAVCNFSM4S5HM222U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJUGI4TOMBZGE2Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>