Editorial overrides: consider marking non-essential parts of a feature

[ddbeck]

We might want to identify features as part of a group, but not let it contribute to a status calculation. This is issue describes a specific kind of editorial override and how we might want to handle it.

Problem description

Some features are plausibly part of a group, but contribute to misleading statuses.

It's likely that some feature members meet the following criteria:

1. It is genuinely part of a wider feature group (i.e., developers wouldn't recognize it as independent feature).
2. It was introduced to one or more browsers around the same time as the majority of the other members of that feature (i.e., web developers wouldn't recognize it as a later addition to a feature — its history starts around the same time as the history of the feature as a whole).
3. Its data uniquely holds back the status of the wider feature (e.g., but for this one member, the wider feature would have Baseline status).

It's likely that we will want to overlook such features for the purposes of calculating a status, to avoid producing a dubious result (e.g., my initial calculation for fetch() reported that it is not Baseline, even though it's a widely-used web platform feature).

These are likely candidates for editorial overrides. But it's already a common enough occurrence that we probably ought to have some tool to handle these cases, rather than resorting to the blunt instrument of simply dictating a status. Plus, it would be desirable to not completely untether the feature from the underlying data.

Candidates for this kind of treatment

• Minor corners of a large feature group (e.g., a missing, little-used property or method)
  • Example: fetch — api.Request.body — fetch: add feature and set Baseline status #478 — not implemented in Firefox
  • Example: web-animations — api.KeyframeEffect.composite — web-animations: revise feature list and set Baseline status #467
• At-risk features
  • Example: offscreen-canvas — api.OffscreenCanvasRenderingContext2D.commit — Features O-R: set Baseline status #462 — proposed for removal from the spec, but not actually removed
• Incorrect data from BCD (e.g., an incorrectly un-mirrored feature)
  • Example: :read-only and :read-write pseudos — Features O-R: set Baseline status #462 (comment) — faulty Firefox for Android data in BCD (at the time I'm writing this)

Potential approaches

Instead, we might consider flagging certain features as being non-contributors to status calculations and perhaps set up tooling around those features, so that those exceptions are re-reviewed when underlying data changes. Some options include:

• pragma comments or annotations in feature lists
• ignore lists per feature
• an inventory of no-op BCD keys

[tidoust]

Is "Incorrect data from BCD" a good candidate for this kind of treatment, though? It seems easy to fix BCD data, and actually good to have an incentive to fix the data early on. The fix to BCD data should also not take long given the proximity of both projects, and flagging features as baseline does not seem to be something that needs to be done urgently.

pragma comments or annotations in feature lists

Yes, I think it would be useful to make explicit what features got excluded and why (although the why is probably going to be "corner case feature" in most cases, and can perhaps be omitted when it is). Data consumers might want to list these features somewhere so that it's clear for developers that they were excluded on purpose.

For features that are proposed for removal but not yet removed, there should be an open issue somewhere about that. Linking to it would allow to detect when that is done. We follow a similar rule when we patch data in Webref: we report the problem to the underlying spec and include a link to the created issue/PR in the patch. A job then alerts us in case a patch is no longer needed.

[captainbrosset]

Generally, I agree that this ability is needed. But I'm worried about this particular case:

Minor corners of a large feature group (e.g., a missing, little-used property or method)

Candidly, if a method/attribute/piece is really part of the feature, then its status should really count towards the entire feature, like the other pieces of the feature.

Giving the governance group the ability to arbitrarily decide to remove some pieces feels risky. How will the group make the decision? How will the group know that a piece isn't being used much? Maybe it's not used because it's not interoperable in the first place.

If we do this, I would suggest considering:

• Documenting the decision, maybe directly within the feature data, as a field of the YAML file that could be exposed in the UI.
• Naming the group in a way that makes this visible. E.g. FeatureXWithoutFooPiece would be a group that's high baseline but that doesn't contain Foo. Versus FullFeatureX which might not be baseline, but would contain each and every little detail of the feature X.

Using data like this, baseline statuses that appear on websites out there would be able to show what, exactly, does a feature contain. Or, more important, what it doesn't contain. Like Fetch, baseline 2023 (except Request.body) (just making up an example).

I would hate for this to become an escape hatch for a browser vendor that doesn't want to implement one little missing thing for whatever reason, when developers out there really need it, but haven't been able to express it loudly.

In my ideal world, stuff like this would be easily visible and linked to each browser vendor's bug tracker so that people can ask for it.

What do you think?

[tidoust]

Giving the governance group the ability to arbitrarily decide to remove some pieces feels risky. How will the group make the decision? How will the group know that a piece isn't being used much? Maybe it's not used because it's not interoperable in the first place.

Right, still building on our experience with patching data in Webref, a key reason why we try to make sure that there is an issue open against the spec, and that we link to it from the patch, is that there may be different ways to fix the problem, and we don't feel that we should choose one: the group responsible for the spec has the authoritative power. In such cases, we try to err on the conservative side of things while the group resolves the issue, e.g. rolling back the data in Webref to what it was before the problem was introduced.

Back to features, if the feature was introduced or changed recently (not what this issue is about, I know), we're dealing with an upcoming feature and the problem seems more a naming/versioning issue. E.g. what Media Source Extensions (MSE) means today vs. what it will mean in a few years from now ("MSE in workers" was introduced more recently. The upcoming ManagedMediaSource interface is not even yet in the spec. Both features should be part of MSE down the road).

If the feature has been around for some time, it signals an issue with the spec. And I think the issue needs to be raised with the working group in charge of the spec and with implementers before it gets dropped from the status calculation. We're already used to doing that, I think (see #462 (comment) for the investigation on commit()), I'm just suggesting to continue!

For Fetch, I note that there have been renewed discussions last year on request.body that resulted in updates to the spec (to drop support for the feature when HTTP/1.1 is used), and concluded that the feature should be kept around. In other words, this should probably still be treat as an upcoming feature. It seems fine to call "Fetch" today is "Fetch without request.body" in the same way as we would define MSE as "MSE without ManagedMediaSource".

In my ideal world, stuff like this would be easily visible and linked to each browser vendor's bug tracker so that people can ask for it.

+1 to linking to relevant places. To bug trackers and/or to an issue raised against the spec itself.

Side note: we should probably not categorize the features we're considering here as "minor" (for whom? The Fetch discussion linked above shows that some developers consider the feature as crucial) or "little-used" (of course they are, they're not widely available!). We're more dealing with advanced features that can be considered non-essential in the sense that the global feature is already useful and usable without them.

[captainbrosset]

It occurred to me that one possible way forward would be to create sub-groups within a feature file.

For instance, for fetch.yml:

spec: https://fetch.spec.whatwg.org/
caniuse: fetch
initial_fetch:
  status:
    baseline: true
    support:
      chrome: "105"
      chrome_android: "105"
      edge: "105"
      safari: "10.1"
  compat_features:
    ...
later_fetch:
    status:
    baseline: false
    support:
      ...
  compat_features:
    ...

Instead of deliberately hiding pieces of a feature because they're not implemented somewhere, and we really to make a feature be baseline, we could always list every little detail of a feature, but sort them into logical groups.

initial_fetch and later_fetch are terrible group names here of course. But think of it as the initial fetch baseline which made it possible for people to use fetch in like 99.9% of the times. There's more to the spec, and browsers add additional pieces over time.

[tidoust]

Discussed in last week's WebDX CG baseline meeting.

General feeling in the group is that features "excluded" from a feature group should be flagged somehow. There are different ways to do this. One idea raised would be to create specific groups for them right away to:

• record why they were left out of the other feature group
• create an identifier that can be used to talk about them