RFC: Self-hosting - Use generated protocol models and metamodels

[akphi]

💬 Request for Comments

We are a modelling tool, in fact, our protocols and metamodels are models we could build in Pure as well, and worse yet, we are not auto-generating these models but repeating their definition in 3 code bases legend-studio, legend-engine and legend-pure. We should model these in Pure and code-gen their definition into Typescript for consumption in Studio codebase.

Simplify metamodel

We need to simplify metamodels so that these can be auto-generated from models in Pure. After this, we're sure that the structure of metamodel more consistent and easier for code-generation

• Move the lambdaId to stores - Stabilize relational mapping editor and compilation error handling in form mode #227
• Move PackageableElementSelectOption back into stores, don't leave those in graph and metamodels
• Remove rendering logic like label, e.g. MappingElementLabel - optimize path and element resolution when building graph #1068
• For fields that we added to support better graph navigation (i.e. owner, parentClass, etc.) that are not in the actual metamodel, we should prefix them with _, such as _owner` - simplify metamodels #1153
• Move helpers logic into corresponding classes MappingHelpers, PackageableElementHelpers like in finos/legend-pure - remove Stubable and more metamodel simplifications #1159
• IMPORTANT Add field integrity verification to hashCode (basically check if properties annotated with ! are actually non-null) - Detailed in Feature request: Metamodel verifier #288 but no longer deem needed
• Consider to remove unnecessary interfaces (these make it hard for tools like VSCode to know where the interface method being called):
  • Stubable interface -> Have a factory to create stub and check for stub maybe? -> move the logic out for stubable to a factory or some helper methods to initialize new instances - remove Stubable and more metamodel simplifications #1159
  • Hashable interface, we don't need models to implement this interface anymore. - see below
• Move validation logic to top-down as well - we will most likely do this later in Feature request: Support global linter/validator #1168
• For certain @computed cases (e.g. allSuperClasses, allSubClassses) we might not need that as computed value at all - remove Stubable and more metamodel simplifications #1159
• MAYBE Turn ALL metadata fields (fields like Property's owner, uuid etc.) to readonly and leave the access modifier as public - ONLY during initialization do we allow to change these readonly property using the Writable improved mapped type modifiers trick in Typescript, write a good doc there to explain why this abuse should only be treated as a hack used ONLY during graph initialization. Or we can simply use Object.assign syntax... - simplify metamodels #1153

type Writable<T> = { -readonly [K in keyof T]: T[K] };

NOTE: there are fields that the UI does not and perhaps should not allow changing like class to a ClassView, but that's an UI decision, this field is not metadata, and it should not be made readonly

• Remove all constructor parameters. Then remove all initialization logic from constructor, in fact, we could remove/make private the constructor all together and replace them by static creator methods
  • Do not initialize any value, as these should be taken care of by the static creator method
  • Use ! and ? in the property definition
  • Remove constructor
• Move hashing logic out of metamodels:
  • After we move hashing and validation logic, revisit spots where we should use isStubbed_... method and use them, right now to avoid bad dependencies between models and the helpers method, we don't do this
  • Move remaining code-logic in DSL Diagram classes out to helpers.
  • Consider moving logic in Packageable Element out
  • Use the following technique to move hashing out. Remember that this is useful for derivations, which are the hardest things left to move. Hashing is one example of derivations which is useful to us and we should keep around. Also, if we don't do hashing at protocol level, I think doing hashing in the reactive manner is still the best, but we will see

// ModelExtension.ts
declare module './models/metamodels/pure/packageableElements/PackageableElement' {
  interface PackageableElement {
    get path(): string;
    get hashCode(): string;
  }
}

...

export {}; // export this so `Typescript` --isolatedModule check does not complain

// --------------------------------------------------------------

// Some file that we call once when we load the app
export const decorateMetamodel = (): void => {
  // e.g. Constraint
  Object.defineProperty(Constraint.prototype, 'hashCode', {
    get: function (this: Constraint) {
      return hashArray([
        HASH_STRUCTURE.CONSTRAINT,
        this.name,
      ]);
    }
  });
};

Other considerations

• CONSIDER We probably would need to take care of visitors as part of code generation. JS does not support overloading like Java so we can't really have 2 accept() methods, we would need to have them differentiated, e.g. accept_SetImplementationVisitor() and accept_PropertyMappingVisitor() for embedded property mappings.
• CONSIDER How do we support Typescript union type? Maybe we can use Any in Pure and then use a tagged-value to annotate union type?

Class/Interface swap:

Since Pure support multiple inheritance, we might need to do what legend-pure does by turning metamodels into interfaces to support multiple-inheritance (with C3 linearization support?), something like the following

• Create CoreInstance interface and ConcreteCoreInstance class:

interface CoreInstance {
  /**
   * Can be used to uniquely identify the instance in a collection (used for UI majorly)
   */
  _UUID?: string;
}

class ConcreteCoreInstance implements CoreInstance {
  _UUID?: string;
}

• We should turn each class into _Impl (that extends ConcreteCoreInstance) and have everything as interface that extends CoreInstance: i.e. Class vs ClassImpl.
• MAYBE We should have optional UUID to each CoreInstance, add a method to allow setting this UUID. NOTE that if we make this non-optional, we're adding a lot of data to the graph
• At this point, we lose the ability to do instanceof, to recover, for each interface, we can do the following:

// Class.ts
export interface Class {
  ...
}

export const Class_InheritanceSignature = 'Class';

// Class_Impl.ts
class Class_Impl implements Class {
  ...
}
Class_Impl.inheritanceSignature = Class_InheritanceSignature;
Class_Impl.prototype.superTypes = [PackageableElement_InheritanceSignature, CoreInstance_InheritanceSignature];

const instanceOf = <T>(val: unknown, inheritanceSignature: string): val is T => val.protoype.superTypes.includes(inheritanceSignature);

interface A {
  type: "1";
}

interface B {
  type: "2";
}

interface C extends A {}

class D implements C {
  type: "1" = "1";
}

class E implements B {
  type: "2" = "2";
}

const instanceOfA = (val: Record<PropertyKey, any>): val is A => {
  return "type" in val && val.type === "1";
};

const invD = (d: D): void => {
  console.log(d.type);
};

// NOTE: this line is perfectly fine in Javascript, but during compile time, Typescript will complain
invD(new E()); // Argument of type 'E' is not assignable to parameter of type 'D'