Data Components Standard for Enterprise scale

This document describes data handling method to be applied on @fundamental-ngx/platform level in order to simplify development cycle for accessing the enterprise resources (such data).

There are many component on enterprise level that works with large set of data such as:

• Datatables
• InputSearch
• ComboBox (Autocomplete)
• Tree
• List
• and much more...

and for those we need to unify the way we access our data as we leave most of the complex data access or manipulation on application which leads different implementation approaches and allot of duplication.

Regardles whether we work on Front-end or backend the principals are identical, therefore I will try apply some of them in here.

Context

It is common practice when working with different data input sources we use e.g. Adapter Pattern that provides us with guidelines how different interfaces can talk to each other but also from here we were able to derive another pattern so called Datasource pattern which is high-level architectural pattern using adapter pattern and some other ones depending on the usage (Proxy, Interceptor, you name it) to really abstract the way how we access internal resources (DB calls, Rest calls, domain objects) with a mininal amount a code.

What we want to achive is to define abstract layer where each specific resource needs to implement and move all the related logic into reusable DataSources.

Current Solution

In normal scenario application developer usually do following e.g: :

Let's use component from a libray:

<w-autocomplete placeholder="Search..."  [list]="users" 
                                        (onSelect)="itemSelected($event)" 
                                        (onSearch)="doSearch($event)"
                                        [suggestions]="mySearchResult" 
                                        [mutliSelect]="true">

</w-autocomplete >

To make it all work application developer needs implement all above handlers and much more:

@component({..})
class MySearchPage {

  users: User[]; 
  
   constructor(private userService: UserRestAPI) {
   }

   ngInit() {
     this.userService.load().subscribe((data)=> {this.users = data})
   }
    
   doSearch($event) {
     // another rest api call
     this.userService.find().subscribe((data)=> {this.suggestion = data})
  } 

  // much more logic here
   
}

• Need to introduce complexity into MySearchPage component to do different data manipulation.

  • On application level you don't really want to use users[] directly. Why ? * When developing our rest services we usually use rxjs to subcribe and manipulate our data but what we don't do is; in most of the cases we dont unsubscribe and release any open subsription.
  • Developers needs to start adding unessesary logic to handle selection, suggestions, fetch or any CRUD based operation.

• If we look over several view we are duplicating allot of code.

• When we use detection strategy onPush, we also have to trigger detect changes so our view is aware that our data changed and view needs to be refreshed.

• We need to start working more with observables and be more reactive that removes above pain !

Proposed Solution

Instead of duplicating all this functionality we need to aggregate and extract as much as common functionality into DataSources something we used already 15 years ago. There are certain approches that does not change over time, the only thing that can change how we implement all this. I will use as example Material Design (CDK) as I can see they went similar way.

The whole idea is to hide as much as common logic into this Adapter like class and not to worry about this on the application level otherwise we are not simplifiying we are introducing more problem.

Let's have a common Interface we can call:

export interface DataSource<T> {
   open(): Observable<T[]>;

  close();
}

Each type of component such Autocomplete, DataTable, List,.. needs to provide its own specific implementation to handle specific usecase. Let's see this pseudo code (based on cdk)

export abstract class DataTableDataSource<T> implements DataSource<T> {
  data: T[];
  filterTerm: string;
  paginator: Paginator | null

  filterPredicate: ((data: T, filter: string) => boolean;  

  sortData: ((data: T[], sort: Sort) => T[]);

  //
  create (t: T): T;
  update (t: T): T;
  remove (t: T): T;   
   
}

// Autocomplete, choosers,..
export abstract class ChoiceDataSource<T> implements DataSource<T> {
  data: T[];
  filterTerm: string;
  resultLimit: number;

  filterPredicate: ((data: T, filter: string) => boolean;  
   
}

After we define some structure we can start creating concrete implementation based on the usage or resource

// Autocomplete, choosers,..
export abstract class RestChoiceDataSource<T> extend DataChoiceDataSource <T> {
  // We can use a registry that is able to map a Entity Type = Endpoint
  // have identical way how to fetch , search ,...   
}

Then on application level we dont really need to do all this heavy lifting and just use it like this:

<w-autocomplete placeholder="Search..."  [dataSource]="userSource">
</w-autocomplete >

All this search triggering, input monitoring would happen internally (hidden) and there would be clear interface between data component and DataSource. Then our component can be simplified to:

@component({..})
class MySearchPage {

  userSource: ChoiceDataSource; 
 
   constructor(private registery: DataRegistry) {
   }
   ngInit() {
     this.userSource = registry.queryDataSource(User, DataSource.Layer.REST);
   }       
}

We don't have to even deal with dataSource in the view, let the component to get its data directly.

• Again you can have your own instance of DataSource or you would use Common one
• You could provide API for DataSource Registry, where on application level we would configure a mappings
  • If you work with Entity User -> Use -> this DataSource,
  • For Invoice -> use -> this DataSource
  • etc...

And you can end up with something like this:

<w-autocomplete placeholder="Search..."  [type]="dataType" (onSelection)="OnlyInterestedOnSelection($event)">
</w-autocomplete >

We need to ask:

• What is the minimum amount of code I need to write to get something working ?
• How to provide components which can boost productity ?