Support for Ion Serialization Format

[scottsom]

Since Elasticsearch already accepts JSON, YAML, Smile, and CBOR then I'd like to submit Amazon Ion for consideration. To implement a new XContentType, we just need the Jackson bindings which are available from jackson-dataformat-ion.

• Like CBOR, Ion is more space efficient than JSON
• Ion's optimized for read format should help with source filtering (need to benchmark)
• Ion's typing can make guessing the field type more reliable
• Ion's annotations are useful for storing meta fields
• An Ion XContentType could potentially use a user provided symbol table (via settings) to further improve space efficiency of _source storage
  • Would have to evaluate how much value this would add considering block level compression is already being used

However, I see that you guys are already on the fence about supporting 4 different types as discussed in #22811.

I can work on a patch and some benchmarks but before I get too invested, I wanted to check on the likelihood of it being accepted considering the previous discussion.

[jasontedor]

I suspect that we are unlikely to accept this as it means:

• yet another format to support (with a similar use-case to CBOR) and as you note, we are already leery of supporting many formats
• another runtime dependency

I will leave this open for others to opine.

[scottsom]

Thanks. I can certainly understand and appreciate wanting to minimize the dependency footprint.

My follow up question would be whether you are open to making this customizable via plugins? (providing your own XContentType)

[rjernst]

Since I would love to move some of those content types out to modules/plugins, I think making the content type impls pluggable would be great. I do expect it would take a bit of work, and I don't think it is a priority right now. @scottsom Are you interested in working on this? It would most likely involve a few PRs to clean some things up, in addition to the actual pluggable part. I would be happy to work through this with you.

[clintongormley]

Discussed in FixItFriday - would be good to make our existing formats into modules.

[scottsom]

@rjernst Don't think I can work on this one right now but I may re-visit this in the future. Thanks.

[nik9000]

This has sat open for most of a year without a comment. We've got a ton to do so I don't think we'll be working on making x-content types pluggable any time soon. I think if someone gets interested in this in the future we can reopen it and discuss again, but for now I think the right thing to do is close this as "won't fix any time soon".