Build-in integration with rails/ujs

[seanpdoyle]

As an alternative to building support for Rails' Unobtrusive JavaScript
into @hotwired/turbo itself, instead publish a
@hotwired/turbo-rails/ujs file to re-use the existing @rails/ujs
hooks, and bridge the gaps between new turbo:-prefixed and
ajax:-prefixed events.

This is a re-imagining of hotwired/turbo#40 and
(hotwired/turbo#384). If deemed viable, this work would yield some
follow up tasks:

• drop support for [data-turbo-method] (hotwired/turbo#277)
• drop support for [data-confirm] (hotwired/turbo#379)

[seanpdoyle]

Since this PR is currently a draft, the implementation is coded directly into the Dummy application.

If this approach is deemed viable, we can both build out more System Tests and figure out a way to integrate a new @hotwired/turbo-rails/ujs JavaScript module into the build tooling.

[KonnorRogers]

Hey Sean! Cool to see work done to preserve UJS, I dont mean to hijack this issue, but I think these functions may be worth adding into a Rails-UJS rewrite I created that is intended to be as backwards compatible as possible. I think these functions could fit in nicely with the MrujsTurbo plugin plugin that exists in Mrujs today. Currently it only exists to auto-render TurboStream responses, but im realizing now there could be more functionality like this baked into it. I'm not sure where I'm going with this other than to say I'm going to grab these functions and add them into Mrujs' Turbo plugin.

[seanpdoyle]

@ParamagicDev that's great to hear! I'm hoping that the "interface" we'll end up with will accept a Rails-like interface.

Currently, the Rails object from UJS exposes these selectors and methods as properties. If mrujs could expose those same properties and match the interface, swapping one for the other should "Just Work".

Does mrujs have an "API" outside of a .start() and HTML data- attributes?

[KonnorRogers]

@seanpdoyle it does! In fact I need to work on exposing it. Right now mrujs exposes these under mrujs.querySelectors.<SelectorName>.selector , which is less than ideal in terms of interfacing.

https://github.com/ParamagicDev/mrujs/blob/e83c192b82df25f60a3a3b796e95db698139c9ec/src/utils/dom.ts#L41-L72

I tried my best to keep the data-* attributes the same, however, it appears I missed the mark in how every function was surfaced top-level in Rails-UJS.

[KonnorRogers]

The other issue is that the idea of "delegate" doesnt exist in mrujs. Mrujs has a document-level MutationObserver that fires when attributes change and when new nodes are added to the DOM and then attaches behavior to anything that fits the querySelector parameters. I think this should be easy enough to port however.

[leastbad]

Does mrujs have an "API" outside of a .start() and HTML data- attributes?

https://mrujs.com/

@ParamagicDev is humble but he has really gone above and beyond with mrujs. Being as close to "drop-in" backwards compatible - while allowing for progress wrt things like XHR -> fetch - is a primary design criteria.

He's taken great care to be highly responsive to requests and put a huge amount of energy into making sure the docs are A+.

It's already got a growing userbase with over 2k/week downloads, and a 24/7 support channel.

It would be nice to see Rails core embrace this as an exciting path forward.

[dhh]

I don't see these functions as something fundamentally about UJS, but rather ways of controlling the flow of links and forms, which in my opinion fits well with the overall mission of Turbo. Meaning that these functions are generally valuable and inline with the goals for Turbo, and therefore should live in Turbo. The turbo-rails setup should be as slim as possible, just adding specific bridging to things that are uniquely Rails. I don't think neither data-turbo-method or data-turbo-confirm are uniquely Rails.

[leastbad]

Agree with your outlook and conclusion, for sure.

My comment is really an expression of wishing that Rails itself, as an entity, would do more to promote exciting developments and ideas that emerge within the community - especially ones that aren't on any kind of inclusion track for Rails proper.

When someone in the Laravel community really knocks it out of the park, Laravel really puts its weight behind it. Big idea modules go on the homepage and they presumably work it into their overall project marketing, socials, maybe even documentation. It's omakase... but with the waiter suggesting that you've just got to try the steak.

Whether it's Haml, ViewComponent, StimulusReflex and CableReady, and now mrujs... it would be such a huge source of optimism and energy for the folks who work hard to make sure Rails kicks ass going into decade three to see you tweeting about things you might not plan on using, but think that some of your Twitter followers might love. For there to be excitement from the core team that someone would recognize a hole and spend months making mrujs.

How amazing it would be to wake up and see DHH tweeting about something you poured your heart into instead of quietly wondering if you'll get put on blast for self-promotion if you post about it on the forum. We laugh, nervously, but trust me: I'm just vocalizing what others are shy about saying.

[dhh]

I think it's wonderful to see that we have a wide array of options for substitutions and variations available to the default answers in Rails. And sometimes those options blaze a trail that then inspires adoption in Rails itself, even if it's in a different form.

I also think we should have a proper forum where people who are working on such variations can feel good and safe about pointing to them.

The Rails tent is so large that we could never all agree on a single omakase menu that everyone will love. Plenty of other examples out there, like minitest vs rspec. But the approach that Rails is taking is that there is one original omakase menu that includes a finished set of answers that someone can take to build their apps, without having to configure external packages.

In this particular case, -confirm and -disable are concepts that are core to what Turbo tries to do. Just like Turbo included handling of forms and data-turbo-method links, which were before elements split between UJS and Turbo.

[dhh]

As to this specific PR, I could imagine having a shim that essentially makes the Turbo versions of -confirm and -disable accessible through the old bare options like confirm:. So someone with an existing app could just take out rails-ujs and drop in Turbo and not have to rewrite those options.

[seanpdoyle]

TL;DR

The changes proposed in this pull request are intended to serve as an official
Hotwire package that provides @hotwired/turbo and @rails/ujs backwards
compatibility to give teams the tools to temporarily bridge the gap between
their UJS applications and new capabilities provided by Turbo and Stimulus.

UJS-provided ajax: events (for example, ajax:before, ajax:complete, etc)
are completely replaced by turbo: alternatives.

There is an opportunity to cover the same ground as UJS' other data-
attribute-driven features (and improve some of them!) without 1-to-1 matching
interfaces and parity.

---

The turbo-rails setup should be as slim as possible, just adding specific
bridging to things that are uniquely Rails.

If I had had hotwired/ organization write permissions, I would have created a
turbo-rails-ujs project to serve as its own package. Since I don't have those
permissions, a pull request to merge this code and its tests into main to be
extracted after the fact seems to be the best path forward.

I could imagine having a shim that essentially makes the Turbo versions of
-confirm and -disable accessible through the old bare options like confirm:.
So someone with an existing app could just take out rails-ujs and drop in
Turbo and not have to rewrite those options.

This work is purely motivated by backwards-compatibility. The goal is to provide
existing applications with glue to ensure that Turbo, Stimulus, and UJS co-exist
in harmony.

If an team is planning on migrating an application to Hotwire, code like this
can enable Turbo, Stimulus, and UJS to co-exist without any fundamental changes
to their existing code. In this case, it might even bridge events in a way that
doesn't require any search-and-replace or data-turbo-prefixing.

With that being said, I think it's OK for Turbo's idioms and events to be
incompatible with UJS. Regardless of how I feel about the overlap between
Turbo's and UJS's goals, if the @rails/ujs package is deprecated and slated to
be removed from Rails, any @rails/ujs glue code should prioritize backwards
compatibility over future viability.

Meaning that these functions are generally valuable and inline with the goals
for Turbo, and therefore should live in Turbo.

I agree. I think there's an opportunity to integrate the features provided by
UJS into Hotwire in a way that matches Hotwire's idioms.

For example, Turbo's turbo:-prefixed events cover the same ground as the UJS
provided ajax: events that fired during HTTP request-response cycles. The
introduction of <turbo-stream> elements and MIME type responses replaces and
improves upon UJS's execution of server-generated JavaScript.

It seems to me that we're all in agreement on that fact. Outside of that
behavior, I think the data- attribute-powered capabilities are where our
perspectives differ.

<button> elements versus <input type="submit"> elements

As some additional context, it's important to highlight the differences between
a <button> element and its <input type="submit"> counterpart.

The biggest fundamental difference is <button> elements can contain descendant
HTML elements (like <svg>, <span>, <b>, etc.) and <input type="submit">
elements cannot.

A <button> element can have and style descendant elements and text while an
<input type="submit"> must declare its button text as a flattened string in
its [value] attribute. By declaring its text content as the [value], an
<input type="submit"> defers any opportunity to embed significant data.

For example, consider an Article posting form. There might be two form
submission elements: a "Publish" element and a "Save as Draft" element.

When implemented as <button> elements, we can encode the Article's state
directly into the submitter's [value] attribute. When paired with a [name]
attribute, the [value] is serialized by the browser and submitted to the
server as part of the HTTP request:

<form method="post" action="/articles">
  <!-- the rest of the form -->

  <button name="state" value="published">Publish</button>
  <button name="state" value="draft">Save as Draft</button>
</form>

If we wanted to add different icons for each element, that could be achieved by
embedding <svg> elements:

<form method="post" action="/articles">
  <!-- the rest of the form -->

  <button name="state" value="published">
    <svg><!-- ... --></svg>
    Publish
  </button>

  <button name="state" value="draft">
    <svg><!-- ... --></svg>
    Save as Draft
  </button>
</form>

If we wanted to styled those <svg> elements based on hover or focus state, we
could declare CSS rules to do so:

<style>
  svg                         { fill: currentColor; }
  button                      { color: black; }
  button:hover, button:focus  { color: blue; }
</style>

If we wanted to style particular words or add additional visually hidden text
that could be accessible to assistive technology, that could be achieved by
wrapping them in HTML elements:

<form method="post" action="/articles">
  <!-- the rest of the form -->

  <button name="state" value="published">
    <svg><!-- ... --></svg>
    Publish <span class="sr-only">to everyone</span>
  </button>

  <button name="state" value="draft">
    <svg><!-- ... --></svg>
    Save as <b>Draft</b>
  </button>
</form>

If the submitters were <input type="submit">, things are much more complicated.
For example, to declare submitters with "Publish" and "Save as Draft" text,
those values would need to be embedded directly into the element's [value]
attribute.

<input type="submit" value="Publish">
<input type="submit" value="Save as Draft">

This means that they're unable to declare a meaningful [name] and
[value] pair to encode additional information about the submission. That means
that if we want to declare two buttons with different behaviors, we'd need to
work around that limitation. That could be worked around by adding JavaScript
behavior, but using a <button> and directly depending on browser-provided
behavior has the potential to be maintenance-free.

Similarly, since <input type="submit"> elements cannot have descendant HTML,
they cannot have <svg> element icons, and cannot vary the style of the
element's rendered text. They can declare CSS rules to style the text, but that
style is applied uniformly to each character. They could also embed icons as
part of a background-image: or ::before { content: ... } CSS rule, but that
requires work during the asset building step.

Even if those icons are embedded, they cannot be styled by the CSS applied to
their ancestors, browsers would not apply ancestor CSS rules to the icons, which
means that we'd be responsible for managing different background-image: URL
values for each possible combination of states we'd want to style.

I think as Hotwire's patterns and idioms mature, there's an opportunity to
establish a preference for <form> elements with <button> submitters, and to
build-in features that depend on and leverage that preference.

Porting [data-disable] and [data-disable-with]:

I've opened hotwired/turbo#386 to propose that Turbo should set a form's
submitter (a <button> or <input type="submit">) element's [disabled]
attribute when a form submits, and then remove it when the submission receives a
response.

The UJS way of embedding text into a data-disable-with attribute works for the
most simple use cases for both <input type="submit"> elements and <button>
elements. However, that style has its own shortcomings in addition to the
shortcomings of the <input type="submit"> element discussed above. For
example, it does not support HTML elements of any kind, regardless of whether
its declared on an <input type="submit"> or a <button>.

Holding Turbo responsible for managing the [disabled] state directly and
automatically, opens the door to JavaScript-less versions of the text toggling
that UJS provides. For example, hiding and showing elements with CSS based on
their :disabled pseudo-class (try it out it out in Firefox or Chrome):

<style>
  button          .show-when-disabled { display: none; }
  button:disabled .show-when-disabled { display: initial; }

  button          .show-when-enabled  { display: initial; }
  button:disabled .show-when-enabled  { display: none; }
</style>

<script>
// There is a recent Safari bug related to SubmitEvent.submitter
//
// https://bugs.webkit.org/show_bug.cgi?id=229660
addEventListener("submit", (event) => {
  event.preventDefault()
  
  event.submitter.disabled = true
  setTimeout(() => event.submitter.disabled = false, 1500)
})
</script>

<form method="post" action="/articles">
  <!-- the rest of the form -->

  <button name="state" value="published">
    <span class="show-when-enabled">Publish</span>
    <span class="show-when-disabled">Publishing...</span>
  </button>

  <button name="state" value="draft">
    <span class="show-when-enabled">Save as Draft</span>
    <span class="show-when-disabled">Saving as Draft...</span>
  </button>
</form>

Similarly, if we wanted to style the <svg> elements differently when their
<button> ancestor is disabled, we could target them with CSS (try it out it out in Firefox or Chrome):

<style>
  svg                                 { fill: currentColor; }

  button                              { color: black; }
  button:disabled                     { color: gray; }
</style>

<script>
// There is a recent Safari bug related to SubmitEvent.submitter
//
// https://bugs.webkit.org/show_bug.cgi?id=229660
addEventListener("submit", (event) => {
  event.preventDefault()
  
  event.submitter.disabled = true
  setTimeout(() => event.submitter.disabled = false, 1500)
})
</script>

<form method="post" action="/articles">
  <!-- the rest of the form -->

  <button name="state" value="published">
    <svg><!-- ... --></svg> Publish
  </button>

  <button name="state" value="draft">
    <svg><!-- ... --></svg> Save as Draft
  </button>
</form>

If Hotwire were to encourage the usage of <button> elements over <input type="submit"> elements, I see that PR as an opportunity to break with the
existing data-disable-with (or data-turbo-disable-with) attribute API in
favor of HTML and CSS that's more robust, relies on browser-provided
capabilities, and can be adjusted to match specific applications needs.

In the meantime, building-in support for the existing @rails/ujs version of
[data-disable] and [data-disable-with]could buy teams time to migrate their
existing <input type="submit"> elements to <button> elements. If teams are
committed to migrating off UJS, there's an opportunity for that effort to
yield new capabilities.

I see a search-and-replace migration of data-disable with data-turbo-disable
as less desirable to temporarily depending on a package of glue code built
specifically for backwards compatibility.

Porting [data-confirm]

The scope of the UJS' data-confirm is extremely tight. What follows is a
functional implementation (try it out it out in Firefox or Chrome):

<script>
addEventListener("click", (event) => {
  const element = event.target.closest("[data-confirm]")
  const prompt = element ? element.dataset.confirm : null

  if (prompt) {
    if (confirm(prompt)) return
    else event.preventDefault()
  }
})
</script>

<a href="https://hotwired.dev" data-confirm="Are you sure?">Visit hotwired.dev</a>
<a href="https://hotwired.dev">Yes, I'm sure. Visit Hotwired.dev</a>

<form action="https://hotwired.dev">
  <button data-confirm="Are you sure?">Visit hotwired.dev</button>
  <button>Yes, I'm sure. Visit Hotwired.dev</button>
</form>

To me, data-confirm is <button>-, <input type="submit">-, <a>-, and
<form>- agnostic. Building it into Turbo itself event feels like a missed
opportunity. Given how it can exist outside of an HTTP or form submission
context, could it exist as a Stimulus extension? Could it exist on its own?
Could it be deferred to user-land as an application-level concern?

<form>, <button>, and <input> elements versus <a> elements

To start, there are accessibility issues with treating <a> elements like
<button> or <input type="submit"> elements. Browsers (and assistive
technology) expect the semantics of <a> elements to involve idempotent
navigations away from the current page.

It seems like think these accesibility issues are already on the team's
radar:

For accessibility reasons, we should continue to promote buttons rather than
method links, but this makes Hotwire much easier to fit into existing
applications that were built to Rails UJS standards.

Thinking outside of accessibility, <a> elements trigger HTTP GET requests,
which have their own idempotent semantics. When rendered with [data-method]
and handled by UJS, it's also important to declare
rel="nofollow"
so that search engines crawling the page don't "click" the <a> and submit a
destructive action in an attempt to ingest the page.

Browsers and assistive technologies expect <button> elements, <input>
elements to manipulate the current page. Like an <a> element, a <form>
method defaults to making an idempotent GET HTTP request when declared without
a [method] attribute. When declared with [method="post"], submitting the
form will result in an HTTP POST request.

If a single <form> element has multiple actions, those difference could be
declaratively stated on their submitter buttons. To override the <form action="...">, declare a [formaction] attribute. To override the <form method="...">, declare a [formmethod] attribute.

Porting [data-method]

To me, continued support (outside of a backwards compatibility bridge like the
proposed @turbo/rails-ujs bridge) of [data-method] and [data-turbo-method]
is a non-starter.

Porting a[data-method]

Consider an <a data-method="..."> element:

<a href="/articles/1" data-method="put" data-params="state=published">Publish</a>
<a href="/articles/1" data-method="put" data-params="state=draft">Save as Draft</a>

Behind the scenes, clicking an <a data-method="put" data-params="...">...</a>
element triggers JavaScript code that constructs a detached <form> element,
sets the [action] attribute to the [href] value, sets the [method]
attribute to the [data-method] value, and de-serializes the value of
[data-params], then re-serializes the values into <input type="hidden">
elements that it embeds into the detached <form> element.

If [data-method] is declared as an HTTP verb other than "post" or "get",
UJS sets the synthesized <form> element's [method="post"], then inserts an
<input type="hidden" name="_method" value="..."> element. The code that
synthesizes the <form> element is also responsible for creating and inserting
elements that embed CSRF tokens when applicable.

When accounting for <turbo-frame> navigations, it's important where in the
document the <form> is mounted, so that must be managed as well.

Once constructed and inserted into the document, the <form> element is
submitted programmatically.

If integrated with [data-disabled] or [data-disable-with], the <a> must
also be "disabled" and have its contents swapped out for the value of
[data-disabled-with]. Unfortunately, <a> elements cannot be "disabled" in
the same way that <input> and <button> elements can declare [disabled], so
the UJS implementation is responsible for re-implementing that behavior and
toggling it on and off during the HTTP request-response cycle.

This process, to me, is client-side templating in sheep's clothing. Taking on
the responsibility of re-implementing built-in browser capabilities was a
necessary concession to make when Rails UJS was introduced. In 2021, Hotwire is
presented with an opportunity to reverse that decisions, and shed that burden of
responsibility.

Alternatively, if the server generated HTML were built a <form> and <button>
pairing, Turbo could treat it the same way it does all other <form>
submissions:

<form action="/articles/1" method="post">
  <input type="hidden" name="_method" value="put">

  <button name="state" value="published">Publish</button>
  <button name="state" value="draft">Save as Draft</button>
</form>

This version embeds its destructive semantics directly into the elements
themselves, and would even continue to work in the absence of Turbo (or
JavaScript entirely)! That's the progressive enhancement upside of promoting
browser built-ins like <form> and <button>.

Using <button> elements also provides an opportunity to declare [formmethod]
and [formaction] attributes, which serve as browser built-in alternatives to
declaring [data-method] and [data-url] attributes directly on the element.

The caveat here is that form is required to contain the <input type="hidden" name="_method" value="put">. Luckily, Rails has built-in support for rendering
that on the server-side with its collection of Action View helpers, and relying
upon server-side HTML generation is one of Hotwire's primary tenets.

Porting input[data-method], textarea[data-method], select[data-method]

UJS supports transforming interactions with form fields into HTTP requests.

For example, consider the following HTML:

<label>
  Published
  <input type="radio" name="state" data-remote="true" data-method="put" data-url="/articles/1" data-params="state=published">
</label>

<label>
  Save as Draft
  <input type="radio" name="state" data-remote="true" data-method="put" data-url="/articles/1" data-params="state=draft">
</label>

Similar to handling other elements annotated with [data-method], UJS will
transform the data- attributes into a synthesized <form> element, which
it'll submit.

As an alternative, consider a <form> element annotated with some Stimulus
attributes:

<script type="module">
  import "https://jspm.dev/form-request-submit-polyfill"
  import { Application, Controller } from "@hotwired/stimulus"

  class FormController extends Controller {
    requestSubmit() { this.element.requestSubmit() }
  }

  window.Stimulus = Application.start()
  Stimulus.register("form", FormController)
</script>

<form action="/articles/1" method="post"
      data-controller="form" data-action="input->form#requestSubmit">
  <input type="hidden" name="_method" value="put">

  <label>
    Published
    <input type="radio" name="state" value"published">
  </label>

  <label>
    Save as Draft
    <input type="radio" name="state" value="draft">
  </label>

  <noscript>
    <button>Save</button>
  </noscript>
</form>

Instead of depending on code to ingest the data- attributes and transform them
into a synthetic <form>, this version depends on built-in behaviors and
semantics. Since Stimulus is part of the Hotwire suite of tools, we can depend
on a simple form controller to request that the <form> element be submitted
whenever an input element fires.

We can even embed a <button> element within a <noscript> element to continue
to support the <form> element's submission in the absence of JavaScript, which
isn't at all possible in the UJS version.

Wrapping up

I didn't expect to write a response like this, but once I started to dig into
the details and think through the original goals of this PR's proposed set of
changes, sharing as many examples as possible felt like an appropriate thing to
do.

I think that incorporating hotwired/turbo#386 into Turbo 7.1, creating a
separate @hotwired/turbo-rails-ujs package, and some changes to the
documentation site could edify these idioms and provide an opportunity to drop
built-in support for UJS in future versions.

[seanpdoyle]

@dhh With Rails 7 out the door, and data-turbo-* prefixed versions of UJS functionality baked-into Turbo 7.1.0, I wonder if there is still an opportunity for this applications to utilize this bridge code.

The link_to and button_to helpers still support disable_with:, confirm:, and method: options, which map to [data-disable-with], [data-confirm], and [data-method] HTML attributes.

Since the Turbo versions of these data-attribute driven behaviors are prefixed by [data-turbo-]. This means that new Rails 7 applications (and applications powered by older versions) require passing data- prefixed or data: { .. } nested attributes to use the Turbo versions.

This means that applications upgrading and migrating from UJS need to modify Action View helpers throughout their view layer, removing options with built-in support for data: {} attributes.

If we were to merge this PR (or another like it), the bridge code could enable applications to opt-into Turbo and more gradually remove or migrate [data-*] attribute-powered UJS features.

Right now, the [data-*] and [data-turbo-*] disconnect is a barrier to adoption.

[dhh]

I'd rather try to solve this on the Rails side, tbh. Maybe we can do a shim gem that simply turns those confirm, method, whatever calls into the format needed for Turbo? I suppose we could even consider doing it as part of this gem. A compatibility mode that can be toggled via a config, but that's off by default.