Implement modular storage handlers

[eduardoboucas]

To extend the number of storage handlers supported by CDN, we should introduce a modular plugin architecture that would allow developers to plug in any storage service as a source for CDN, in the same way we built support for API data connectors and Web template engines.

The anatomy of a handler

A storage handler should be published as an npm module (e.g. @dadi/cdn-dropbox). It must expose a pre-defined set of methods that allow the core app to retrieve files and metadata.

Configuration

Configuration for the various storage handlers could be done within a sources (or storage?) block in the configuration file.

{
  "sources": {
    "dropbox": {
      "enabled": "true",
      "apiKey": "123456789"
    },
    "directory": {
      "enabled": "false"
    },
    "s3": {
      "enabled": "false"
    }
  }
}

Currently, we have different configuration blocks for images and "assets" (i.e. everything that isn't an image). I'm not entirely sure assets is the best term to use, but more importantly, do we need a distinction at storage handler level? Do we think people will want to load images from one place and everything else from another place? And if so, shouldn't we think of a more granular way of defining that (e.g. a global storage handler and overrides on an extension/mime type level)?

Initialisation

CDN could ship with a few storage handlers by default (at least HTTP and local disk). Additional handlers could be installed in the same way template engines are installed in Web.

1. Install the package and save it as a dependency

   npm install @dadi/cdn-dropbox --save
   

2. Include the storage handler as part of the app initialisation

   require('@dadi/cdn')({
     storageHandlers: {
       dropbox: require('@dadi/cdn-dropbox')
     }
   })

A penny $DADI for your thoughts @jimlambie.

[jimlambie]

@eduardoboucas I think this is the next logical place to go, for sure. We're close to having Digital Ocean support but that's an easy one to pick off because it's compatible with the S3 API. Breaking this out into pluggable modules is a great move.