Support Schema reference to other Schema files

[johnmorrell]

With complex Schemas it can be useful to define in multiple files and reference across files, eg:

"$ref":"definitions.json#/quantity"

Currently (v2.1.0-rc.0) such references seem to fail to resolve. It would be very useful if this was supported.

[edgarmueller]

Yes, I agree. I think part of this issue is that resolveSchema does not correctly consider cases where the schemaPath points to a ref again. We'll have a look soon and will try to come up with a fix.

[sdirix]

I took a closer look at this issue. JSONForms only resolves references within schemas by design.

JSONForms still supports schemas split over multiple files, however you have to take care of the resolving yourself. For this you need to create your own Ajv instance and execute addSchema on it with your schema dependencies. Afterwards you pass the Ajv instance to the the init action of JSONForms.

Example:

import {
  Actions,
  createAjv
} from '@jsonforms/core';

// ...

const ajv = createAjv();
ajv.addSchema(
  definitionsSchema, // the schema you got from your disk, REST call, etc.
  'definitions.json' // the id which will be used to refer to this schema
);
store.dispatch(
  Actions.init(
    data,     // your data
    schema,   // your schema (which might reference 'definitions.json')
    uischema, // your uischema
    ajv
  )
);

This should already work in your application. You can also check #1241 where we adapted our resolve examples.

Please let us know if this answers your question or if you had another use case in mind.

[johnmorrell]

Thanks.

I have tried this (v2.1.1-alpha.2) and it looks like the ajv is setup correctly with the primary schema referencing the definitions schema, but the app fails to launch with error:

TypeError: undefined is not an object (evaluating 'schema.properties')

index.js

import React from 'react';
import ReactDOM from 'react-dom';
import './index.css';
import App from './App';
import registerServiceWorker from './registerServiceWorker';
import { combineReducers, createStore } from 'redux';
import { Provider } from 'react-redux';
import { Actions, createAjv, jsonformsReducer } from '@jsonforms/core';
import { materialFields, materialRenderers } from '@jsonforms/material-renderers';
import schema from './schema/schema.json';
import definitionsSchema from './schema/definitions.json';
import uischema from './uischema.json';

const data = {};

const ajv = createAjv();
ajv.addSchema(definitionsSchema, 'definitions.json');

const store = createStore(
  combineReducers({ jsonforms: jsonformsReducer() }),
  {
    jsonforms: {
      fields: materialFields,
      renderers: materialRenderers
    }
  }
);

store.dispatch(Actions.init(data, schema, uischema, ajv));

ReactDOM.render(
  <Provider store={store}>
    <App />
  </Provider>,
  document.getElementById('root')
);
registerServiceWorker();

schema.json

{
   "type":"object",
   "properties":{
      "colours":{
         "$ref":"definitions.json#/colours"
      }
   }
}

definitions.json

{
   "colours":{
      "type":"string",
      "enum":[
         "Red",
         "Green",
         "Blue"
      ]
   }
}

If I remove the reference and use the following schema.json the app runs as expected:

{
   "type":"object",
   "properties":{
      "colours":{
        "type":"string",
        "enum":[
           "Red",
           "Green",
           "Blue"
        ]
      }
   }
}

[sdirix]

Thank you very much for the detailed example. We are now able to reproduce the issue and will take a closer look in the next sprint.

[johnmorrell]

Great thank you! We really appreciate how responsive and proactive the team are on this project.

[cliffschwab]

we ran into the same issue, please address it asap. I believe it is a major blocker for any enterprise level application of this framework.

[sdirix]

We are actively working on this issue and try to deliver a solution within this milestone as indicated by our milestone planning.

[johnmorrell]

@sdirix I've just tested this with v2.2.2-beta.0 and is working really well. Thank you for addressing this!

I did encounter one error with a single reference out of 100 or more that failed. I could not see a problem with the schema, so I wondered if there is a limit on the number of recursive references?

I will investigate further and try and provide a repro if it is an actual issue.

[sdirix]

@johnmorrell Thank you very much for already testing the new implementation. We don't know of any limitation but of course there very well could be one we are not aware of.

As it seems to work for the most use cases we will include this fix in the next release and close this issue. If you are able to identify a setup with which the error you are experiencing is reproducible don't hesitate to open a new issue ;)

[cliffschwab]

@sdirix , i tried 2.2.2 release and i run into issues when trying to resolving schema defined in another file, not sure what i missed, vs code validate successfully using testing data, please take a look. Thanks!

here is the piece of schema that refer to the ssn schema defined in commonSchema.json
"ssnCommon": {
"description": "SSN/Tax ID",
"$ref": "commonSchema.json#/definitions/ssn" },

This is common schema:
{ "$schema": "http://json-schema.org/draft-07/schema#",
"description": "Common Schema",
"definitions": {
"ssn": {
"description": "SSN/Tax ID",
"type": "string",
"pattern": "^[0-9]{9}$"}
}
}

Here is the code in index.tsx:

import commonSchema from './commonSchema.json'
import schema from './IRAAO-Schema.json';
import uischema from './IRAAO-UiSchema.json';
const ajv = createAjv();
ajv.addSchema(commonSchema, 'commonSchema.json');
store.dispatch(Actions.init(data, schema, uischema, ajv));

And we got this error:

ono.js:59 Uncaught (in promise) SyntaxError: Error resolving $ref pointer "http://localhost:3000/commonSchema.json#/definitions/ssn".
Token "definitions" does not exist.

[johnmorrell]

There is an additional step to configure the parser via refParserOptions. There is an example here:

https://github.com/eclipsesource/jsonforms/blob/49692fcac1917a0e68efb3ec42f1ba306b95d197/packages/example/src/index.tsx

I did wonder if this could be handled within the store as ajv is given the schema and its reference, so could possibly setup the refParserOptions behind the scenes?

[sdirix]

Thanks @johnmorrell! @cliffschwab currently a user of JSONForms has to configure the 3rd-party libraries we use for validation and resolving themselves.

The init action of JSONForms can be configured to include an AJV instance (for validation) and an options object for JSON Schema $Ref Parser.

Copied (and adapted) from our examples:

  // Add schema to validation
  const ajv = createAjv();
  ajv.addSchema(
    yourSchemaObject,
    'yourSchemaIdentifier'
  );
  // Allow json-schema-ref-resolver to resolve same schema
  const yourSchemaResolver = {
      order: 1,
      canRead: function(file) {
          return file.url.indexOf('yourSchemaIdentifier') !== -1;
      },
      read: function() {
          return JSON.stringify(yourSchemaObject)
      }
  }
  // configuration object for JSONForms
  const jsonFormsConfiguration = {
    ajv: ajv,
    refParserOptions: {
      resolve: {
        foo: yourSchemaResolver
      }
    }
  }
  // Add configuration to JSONForms
  store.dispatch(
    Actions.init(
      data,
      schema,
      uischema,
      jsonFormsConfiguration
    )
  );

---

@johnmorrell We were also thinking about offering convenience actions or utility methods to ease the configuration for resolving. As we didn't want to unnecessarily restrict any user we decided to just expose AJV and the ref-parser configuration to support as many use cases as possible. However I very much like the idea of possibly using the AJV configuration as a fallback when the user did not add a ref-parser configuration. We might visit that and other resolving / configuration topics in the future.

[cliffschwab]

@johnmorrell @sdirix thx for the quick response, it is working now as we expected, great!