(dev/core#4279) Define import-maps for ECMAScript Modules

[totten]

Overview

With ECMAScript Modules (ESMs), one *.js file can import another *.js file. The import statement may use a physical-path or a logical-path. Logical-paths are easier to read, more portable, and more maintainable. This branch adds support for logical-paths.

To try it out, install the esmdemo extension and open page civicrm/esmdemo.

(This is the second part of https://lab.civicrm.org/dev/core/-/issues/4279. It builds on #26195.)

Before

The basic ESM support (#26195) allows you to use import - but only with a physical path.

import { TableWidget } from 'https://example.com/sites/all/modules/civicrm/js/table-widget.js';

After

You may import a *.js file with a logical-path like:

import { TableWidget } from 'civicrm/js/tab-widget.js';

To do so, you must declare the logical-path as part of the "import map":

function my_civicrm_esmImportMap(array &$importMap, array $context) {
  $importMap['imports']['civicrm/'] = Civi::paths()->getUrl('[civicrm.root]/.');
}

The structure of $importMap is defined by the web-browsers. See, e..g.

• https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap
• https://github.com/WICG/import-maps

Technical Details

In the docblocks, you will find discussion about how import-maps should work from a few different perspectives:

• Extension Developer: How to register additional mappings
• Core Developer: How to render a default SCRIPT tag
• UF/CMS Developer: How to integrate Civi's import-map into the UF/CMS import-map.

Comments

This is currently marked draft for a few reasons:

• I need to write+test an example consumer as part of https://github.com/totten/esmdemo/
• It needs some phpunit test-coverage.
• I'd also like to take a stab using the shim for older browsers.
• After we have some working examples, we may want to talk more about the namespace conventions. (What makes for a good or bad import-map?)

[civibot]

(Standard links)

• If you are reviewing this pull-request, you may wish to consult the test sites and the Review Standards (long template, short template).

[totten]

Thanks @eileenmcnaughton. I pushed one commit for all three suggestions.

[totten]

Added a working example on https://github.com/totten/esmdemo/#part-2-import-map-support.

[totten]

@eileenmcnaughton @colemanw - Since importmap only recently hit the milestone of cross-browser support (https://caniuse.com/?search=importmap), I played around with a couple mechanisms to load import-maps on older browsers. Most notably:

• https://github.com/guybedford/es-module-shims: Focused specifically on adding module+importmap support. Adding es-module-shims was pretty easy.
• https://github.com/systemjs/systemjs: An older/more established mechanism with a bigger feature-set and a lot of shared concepts. However, realistically, loading ESM with SystemJS requires a workflow with Babel/NodeJS.
• (Both are written by the same person, who also contributed to the standardization process for importmap. You see a lot of similar ideas in each.)

The next question: looking down the road, what happens if importmap is used in a future version of Drupal or WordPress or Joomla or Backdrop or one of their many addons? And does it make a difference if Civi (or the counterparty) uses es-module-shims? I believe there are a few scenarios for conflicts.

But I think I found a neat way to mitigate the impact of conflicts: make es-module-shims optional. (Admins can set esm_loader=shim or esm_loader=browser). The key thing is that you can force it into shim-mode (where shimmed-modules are separate from browser-native modules). So:

• If (e.g.) WP adopts es-module-shims for importmap, then Civi-WP users can set esm_loader=browser to temporarily side-step a conflict.
• If (e.g.) Drupal adopts browser-native importmap, then Civi-Drupal users can set esm_loader=shim to temporarily side-step a conflict.

Those aren't perfect solutions. (They may require you to trade performance or compatibility; and they're not guaranteed to work in every hypothetical conflict.) But it's a fairly simple adaptation and which plausibly mitigates several contingencies.

So the current revision adds the setting. I also added a Markdown doc discuss more about this.

[colemanw]

@totten

The key thing is that you can force it into shim-mode (where shimmed-modules are separate from browser-native modules)

How would you do that? ES Module Shims seems to act like a polyfill that only kicks in if the browser lacks support for import maps. From the GitHub page:

ES Module Shims entirely bypasses processing for the 74% of users with native import maps support.

Maybe deeper down in the docs is something about forcing it to override the native browser support?

[totten]

(@colemanw) Maybe deeper down in the docs is something about forcing it to override the native browser support?

That's correct. It responds to a few different tags:

<script type="importmap">...</script> <!-- Dynamic polyfill. Use browser-native-mode or shim-mode -->
<script type="importmap-shim">...</script> <!-- Use shim-mode -->

And there's also this clarification. (Note: The README tends to use "polyfill" and "shim" interchangeably.)

... [There] are effectively two loaders running on the page ... if you have two module graphs - one native and one polyfilled, they will not share the same dependency instance...

--

Relatedly - the current revision doesn't enable dynamic polyfill mode, but I wasn't sure about that. Dynamic polyfill mode is probably better balance of compatibility+performance for the typical deployment. (It's just not as useful as an escape-vector in case of conflict.) The only real downside is a longer list of options. ("Browser-native", "Shim-Adaptive", "Shim-Forced").

[colemanw]

The only real downside is a longer list of options. ("Browser-native", "Shim-Adaptive", "Shim-Forced").

IMO the biggest downside to this approach is giving the admin One More Damn Thing To Configure ™; and the only reason they'd ever want to think about it is if their site is broken due to import map conflicts, and in that case, I think chances are very good that the error message they get (if any) would be so obtuse (maybe a 401 buried in the console somewhere) and so weirdly different on different browsers, that they'd never be able to figure out what was going on well enough to know that it had anything to do with this Damn ™ setting.

(Yes that paragraph was a single sentence. I'm gonna trademark that as well.)

So in case that rant wasn't clear, IMO the ideal solution would not involve the admin configuring an obscure setting after tracking down an obtuse error. Ideally we'd be approaching this the way we siloed off jQuery and LoDash with noConflict() mode. Obviously, having duplicate copies of jQuery isn't the most performant choice for every site, but the admin doesn't have to configure anything and it works for every site.

[totten]

@colemanw I definitely share the fear of giving too many options, but I think it's a misplaced here. Let's contrast a few situations:

• When extensions were first added to CiviCRM, there was no default path. Everyone had to navigate to two different settings pages to configure the "Path" and "URL". This was bad. It required extra setup on every deployment (and it was rather unforgiving about the details).
• When writing a builder UI (like "Advanced Search" or "Search Kit" or "Edit Profile" or "New Form"), you have this tension - you can add more functionality to the UX. That requires more widgets. But if you add more widgets, then it becomes more complex for average use-cases.
• "Administer > System Settings > Misc" is a different beast from either of those. It's a list of things that don't matter 95% of the time. They're not important enough to show-off in some place visible. Most people never think about them. But you can't quite remove the 5%. (Or rather, all options for addressing the 5% are bad, and a setting is less-bad in some ways.)

In drafting, I've gone through a progression from:

1. I don't want this to be configurable at all. I just want one thing.
2. Aw shucks, I can't get away with just one thing because of the 5%.
3. Maybe if I sit on my hands for a day it'll go away.
4. Nope, 5% still there. Let's sneak in a define()/constant().
5. Aw shucks, define()s/constant()s aren't as good as settings.

[totten]

Hook signature changed. That's a fairly substantive effect on the description, so I refiled as #26309. Closing since that variant seems preferable.