Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Carryover evaluation: URL #424

Closed
3 tasks done
lucacasonato opened this issue Sep 15, 2023 · 11 comments
Closed
3 tasks done

Carryover evaluation: URL #424

lucacasonato opened this issue Sep 15, 2023 · 11 comments
Labels
carryover-evaluation Evaluate whether to continue a focus area in the next year focus area: URL

Comments

@lucacasonato
Copy link
Member

lucacasonato commented Sep 15, 2023

Description

The URL parsing algorithm in web browsers

Specification

https://url.spec.whatwg.org/

Open Issues

No response

Tests

https://wpt.fyi/results/url?label=master&label=experimental&aligned

Current Implementations

  • Blink
  • Gecko
  • WebKit

Standards Positions

N/A, URL has been widely shipped since the beginning of time :)

Browser bug reports

https://crbug.com/724018
https://crbug.com/1406728
https://crbug.com/1416006
https://crbug.com/1416013

https://bugzilla.mozilla.org/show_bug.cgi?id=1553105
https://bugzilla.mozilla.org/show_bug.cgi?id=1848511
https://bugzilla.mozilla.org/show_bug.cgi?id=1722328

Developer discussions

Confusion because URL parsers across Blink, Gecko, WebKit, Node, and Deno do not interop well. The root cause is nearly always parser bugs in Blink or Gecko: https://twitter.com/oleg008/status/1699087223751073883

Polls & Surveys

No response

Existing Usage

URL is very widely used - custom schemes are commonly used for links to native apps, or when dealing with developer tooling like databases. They may also become exceedingly more common with import maps.

Workarounds

There are no good workarounds - one can import whatwg-url, the URL reference implementation if they need perfectly spec compliant behaviour, but this only gets one so far. This breaks down when using browser built-in features that use URL parsing internally, like import maps.

Accessibility Impact

None

Privacy Impact

None

Other

This has been ab interop problem in browsers for many many years, and except for WebKit, who fixed their parser a couple years ago and proved that a compliant parser is web compatible, no-one has invested the time to fix the shortcomings their browser.

@lucacasonato lucacasonato added the focus-area-proposal Focus Area Proposal label Sep 15, 2023
@foolip
Copy link
Member

foolip commented Sep 18, 2023

@lucacasonato thanks for filing this! As I suspect you've seen, URL is part of https://wpt.fyi/interop-2023. Is this a proposal to carry forward that exactly as-is into Interop 2024, or are you suggesting an expansion of scope?

@lucacasonato
Copy link
Member Author

lucacasonato commented Sep 18, 2023

I'm suggesting to carry it forward because the progress made by Chrome and Firefox was rather limited so far this year. There is still very significant incompatibility, especially around non-special URL handling in Chrome. (this case new URL("postgres://foo:bar@test/db"))

@lucacasonato
Copy link
Member Author

If we wanted to change something to promote more progress in this area, we could consider only including tests that don't already pass in all three engines at the start of Interop 2024. Right now 84% looks "good" only because the majority of tests are for http://, where browsers are already very compatible. By excluding these tests, the focus is put more on the actual incompatibilities.

@hayatoito
Copy link
Member

hayatoito commented Sep 19, 2023

A list of bug reports in Chromium seems outdated.

You can see a list of issues tracked in interop 2023 in Chromium here:
https://bugs.chromium.org/p/chromium/issues/detail?id=1412660

As of Sep 19, there are 4 issues remaining (at least, we are aware of):

  1. https://crbug.com/724018
  2. https://crbug.com/1406728
  3. https://crbug.com/1416006
  4. https://crbug.com/1416013

1 and 2 are about IDNA handling, which no one has started to work on yet.
4 will be fixed in Chrome M119, hopefully, except for 'SPACE' and 'ASTERISK' characters.
So the biggest remaining issue is 3 (Support non-special URL), which I am now investigating.
The impact seems huge.

My rough estimation at present is

  • 1, 2 and (part of) 4 will be carry forward to 2024.
  • I'll update the status of 3 later, which is probably carry over to 2024.

@lucacasonato
Copy link
Member Author

@hayatoito Thanks, I've updated the Chromium issue list. I agree that the impact of the non-special handling in Chromium currently is the largest incompatibility. I don't think we can meaningfully say "URL is interoperable" until this is fixed.

@foolip foolip added the carryover-evaluation Evaluate whether to continue a focus area in the next year label Sep 21, 2023
@foolip
Copy link
Member

foolip commented Sep 21, 2023

I've added the carryover-evaluation label as this is about continuing/expanding an existing focus area.

@jensimmons jensimmons changed the title URL parsing alignment Carryover evaluation: URL parsing alignment Sep 21, 2023
@foolip foolip added focus area: URL and removed focus-area-proposal Focus Area Proposal labels Sep 22, 2023
@lucacasonato
Copy link
Member Author

lucacasonato commented Sep 30, 2023

Just got another report about the non-standard protocol discrepancy on our issue tracker: denoland/deno#20733

@rotu
Copy link

rotu commented Sep 30, 2023

Yup. I was in disbelief that the browsers would be so out-of-alignment on this! My reference of first resort, the MDN page and caniuse, misleadingly show “full support” with no caveats.

@foolip
Copy link
Member

foolip commented Oct 3, 2023

@lucacasonato to make the proposal clear, are you suggesting adding any tests at all, or "just" continuing with the exact set of tests that are part of Interop 2023?

@foolip foolip changed the title Carryover evaluation: URL parsing alignment Carryover evaluation: URL Oct 6, 2023
@foolip foolip mentioned this issue Oct 6, 2023
Closed
3 tasks
@annevk
Copy link
Member

annevk commented Oct 6, 2023

Thank you @lucacasonato for filing this!

In #565 I argued for inclusion of URL as well because of non-special URLs (as you mention) as well as improving IDNA support so end users can reach the same websites reliably. And also that if we were to add #525 as a focus area, it would be especially important to include URL due to URLPattern's fundamental dependency on URL.

Furthermore, I suggested a slight expansion in terms of testing to account for some work I did this year on endpoints that invoke the URL parser with a non-UTF-8 encoding:

@hayatoito
Copy link
Member

Furthermore, I suggested a slight expansion in terms of testing to account for some work I did this year on endpoints that invoke the URL parser with a non-UTF-8 encoding:

Just for the record, the bugs in Chrome are:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
carryover-evaluation Evaluate whether to continue a focus area in the next year focus area: URL
Projects
Status: Done
Development

No branches or pull requests

6 participants