Support for JSON Type Definition

[ucarion]

@epoberezkin, you've voiced interest in implementing JSON Data Definition Format ("JDDF"), formerly known as JSL, in AJV. I'm opening this issue to track that discussion.

It seems to me that the first thing to figure out is the answer to the question:

What would JDDF support for AJV look like?

I think we should figure out the following aspects of the developer experience with using JDDF in AJV, before doing anything else, because they'll define our product requirements:

1. Why would someone choose to use AJV, rather than anything else, for JDDF validation? This should inform everything we do. I think the answer goes back to AJV's core values: excellent performance, excellent conformance, and easy customizability / extensions. But I'd like your take.
2. How do we document support for JDDF in AJV? How do make it so that users aren't confused by whether some feature is JSON Schema-only?
3. How would AJV know whether to use JDDF versus JSON Schema? How does the user indicate which to use?
4. What does writing a custom keyword look like with JDDF?
5. How do we leverage ajv-i18n to support JDDF? This is something that can be easier to support with JDDF -- maybe an AJV-independent package can handle this, since error messages are standardized. i18n error messages for custom keywords is trickier, though.
6. How do we leverage ajv-pack to support JDDF? Is this going to be hard to do?
7. How are features like filtering additional properties, assigning defaults, async loading and validation going to work? These features are pretty tied to particular JSON Schema features.

I'm not suggesting we need a perfect answer to all these questions. I'm saying that these questions inform our requirements. And maybe the first iteration of AJV's support for JDDF is quite limited. That's fine. But we shouldn't go headfirst into implementation if that prevents us from supporting these more advanced use-cases, or if the developer experience is bad.

Once we get better requirements here, I can convert this ticket into a checklist of things we need.

[epoberezkin]

@ucarion thanks for all the questions - it is very helpful.

Firstly, support for JTD is part of the last milestone in Mozilla's MOSS grant, it is planned for Q1'21 - it will be in Ajv v8, so it's about time to answer these.

1. Why would someone choose to use AJV, rather than anything else, for JDDF validation? This should inform everything we do..

My primary objective is to give the existing Ajv users a choice between more complex and expressive JSON Schema, that can be used in a wider range of validation scenarios, and a simpler, more restrictive, and, as a result, less error prone JTD. An additional advantage of JTD is that it is now published as RFC8927, so it may be preferred to some business users.

Why would JTD users choose Ajv - thanks for the compliments about the values, rigour and performance were indeed the main things. I've just made a standalone code generation comprehensive in #1332 (it will supersede ajv-pack in v7), so the users would be able to compile JTD schemas to functions or to standalone validation code that can be used without full Ajv code. I am also considering whether to add schema-based parsers (to fail faster while parsing) - it is not decided yet, possibly next year.

2. How do we document support for JDDF in AJV? How do make it so that users aren't confused by whether some feature is JSON Schema-only?

The docs are restructured, so there will be a separate page jtd.md, alongside json-schema.md, that would give a brief overview of JTD validation primitives. The main page would explain how to choose the correct export to use JTD with Ajv.

3. How would AJV know whether to use JDDF versus JSON Schema? How does the user indicate which to use?

There are already separate Ajv classes (exports) for JSON Schema draft-07 and draft-2019-09 support, so there will be a separate export for JTD too: import Ajv from "ajv/dist/jtd"

Currently the main export (import Ajv from "ajv") points to Ajv class that supports JSON Schema draft-07, and given that it is the most widely used spec it would most likely remain the case in version 8.

All these classes use the same core class, but add different keywords (aka vocabularies) and options, so there will be a documented way to have dual support for both JSON Schema draft-07 and JTD in one Ajv instance, to simplify the migration, should the users need to do it in stages. In this case users will have an option to have a default meta-schema for JTD or for draft-07. Also, while JTD does not define meta-schema to validate its own schemas (it is not needed for most users), there will be a bundled meta-schema for JTD with some additional extension keyword (e.g. anyOf, union or something else).

4. What does writing a custom keyword look like with JDDF?

Ajv v7 uses the same keyword definition API for pre-defined / bundled keywords and for user-defined keywords (I removed the word "custom", as they are all the same with some small exceptions)

"vocabularies" folder here and ajv-keywords package have many different keywords, it will be the same for standard JTD keywords and for any extensions

5. How do we leverage ajv-i18n to support JDDF? This is something that can be easier to support with JDDF -- maybe an AJV-independent package can handle this, since error messages are standardized. i18n error messages for custom keywords is trickier, though.

I didn't yet decide how to simplify extensions inside of ajv-i18n - potentially, they will be run-time so you can add languages for each keyword you define - similar approach that is now used in Ajv for the keywords themselves. There probably will be some mechanism to extend existing keywords definitions to add new functionalities to them - this approach could be used to implement parsing code as a separate extension, so the users who don't need it won't have it in their bundle.

6. How do we leverage ajv-pack to support JDDF? Is this going to be hard to do?

With PR #1332 standalone code generation is now first-class (and it is not included in the browser bundle, so the cost for the users who don't need it is negligible), so it will work with JTD schemas too, as long as all keywords provide code snippets for any shared variables they define with gen.scopeValue. ajv-pack package will not be needed for ajv v7/8.

7. How are features like filtering additional properties, assigning defaults, async loading and validation going to work? These features are pretty tied to particular JSON Schema features.

It has to be decided on a case by case basis. Depending on whether JTD has similar semantics, it can be one of these:

• supported with JTD schemas
• provided as extension to JTD
• not supported

Specifically:

• filtering additional properties - most likely can be supported
• coercion - most likely can be supported
• assigning defaults - could be supported as extension
• async loading - this is orthogonal to the schema spec, but it would require some extension to JTD to support external IDs (which is probably needed anyway)
• async validation - can be supported as extension, it is only meaningful when custom keywords are used - all predefined keywords are synchronous.

No promises that any of those will be supported with JTD in the first release of Ajv v8, but they can be added over time as there is demand.

... maybe the first iteration of AJV's support for JDDF is quite limited ...

I think it should be as comprehensive as draft-07 support, I don't have any big concerns - the new design in v7 makes it achievable. I think it would be equally straightforward to implement any JSON based DSL - I've done it with Ajv before v7, it is much easier now.

[epoberezkin]

Implemented in #1428