Files and Directory Entries API with webkit prefixes!

[cynthia]

Hello TAG!

I'm requesting a TAG review of:

• Name: Files and Directory Entries API
• Specification URL: https://wicg.github.io/entries-api/
• Explainer, Requirements Doc, or Example code: https://github.com/WICG/entries-api/blob/gh-pages/EXPLAINER.md
• Tests: https://github.com/w3c/web-platform-tests/tree/master/entries-api
• Primary contacts: I believe @inexorabletash

Proxy reporting as I believe the TAG should touch on shipping a webkit prefixed spec. I believe they are willing to remove the prefix given that everyone agrees to do it.

[inexorabletash]

Yep, that's me. Thanks for raising this!

As noted in the explainer, the spec attempts to document what's already been implemented and shipping, to help the implementations converge on compatibility with the old, crufty, prefixed Chrome implementation. This seems to have worked - the latest Safari TP added support following the spec guidance and passed nearly all of the tests (which hadn't been upstreamed to WPT at the time).

The four main engines (Chrome, Firefox, Edge, Safari) are now shipping (or on track to ship) with prefixes, for compatibility with Chrome. Adding non-prefixed aliases sounds good to me; I don't know how amenable other implementers will be. (The expression "lipstick on a pig" may be relevant.) Influential feedback from TAG may help.

[annevk]

I'm not sure we should just add nonprefixed variants. I'd rather we first figure out how this integrates with the rest of HTML and then figure out what primitives we have here and what an ideal API would look like. And only then expose that ideal API. If it turns out that's just removing the prefixes, fine, but I don't think we'll know for sure until we do the work.

[triblondon]

I had a read through. Given existing implementations exist, I didn't see anything that made me think it justified the work to change, but...

• the webkitdirectory property of HTMLInputElement allows the selection of multiple files via selecting a directory, but it's not clear whether an <input type='file' webkitdirectory multiple> could permit selecting multiple directories without selecting the parent.

• it doesn't use promises. Should it?

[inexorabletash]

Re: webkitdirectory + multiple

Good point - that needs specifying somewhere. I'll file a bug. In Chrome, at least, webkitdirectory takes precedence over multiple and the latter is ignored if both are specified, so you can't select multiple directories. I have not tested what other browsers implemented.

Re: promises

Since the non-Chrome browsers implemented this API somewhat reluctantly due to market/interop pressure, with entirely reasonable skepticism that the API is an ideal basis for the future, there's also been reluctance to commit to extending it at all, versus possibly introducing a new parallel API design. This manifested, for example, as objections raised at TPAC to adding the non-prefixed directory attribute with the same behavior, holding out hope that it could give access to an improved API instead.

Obviously, per the examples, I'm a fan of promises. :) It would be straightforward to e.g. add the following:

• on FileSystemEntry
  • Promise<FileSystemDirectoryEntry> parent();
• on FileSystemDirectoryEntry
  • Promise<FileSystemFileEntry> file(path);
  • Promise<FileSystemDirectoryEntry> directory(path);
• on FileSystemFileEntry
  • Promise<File> file()

... and a way to get an async iterator for entries on FileSystemDirectoryEntry, e.g. entries().

[travisleithead]

Consulted with @aliams (thanks!)

Our position is that removing the prefix will require the web to make a change; if the web needs to make a change, we would strongly prefer they instead migrate to a better (more ergonomiclly well-designed approach) than this Entries API.

Issues with this API:

1. The technique for getting entries from <input> elements is synchronous (blocking). We actively see performance traces with this on the stack. It can become problematic because the API must aggregate deep directory structures for entries. It also becomes slow if there are a lot of files in a single directory on your local client.
2. Poor ergonomics: there's a weird indirection to get a File from the Entries objects
3. Poor ergonomics: folder path information was added to the flat-list structure. This didn't mesh well with developer's intuitive method for an iterative or recursive approach to "traversing" directories.
4. Poor ergonomics: no use of Promises for async stuff :-(
5. The Entries API has "bolted on" access points that spread into a variety of other objects (e.g., has to extend File to provide *getEntries access points).
6. Empty folders weren't enumerated (didn’t provide an entry). For customer scenarios where the site wants to clone a local directory into the cloud, empty directories will be missing.

Basically, this feature was hobbled together from partially-implemented parts of an older API, plus a adding a set of extensions as documented in a Wiki. It's not really fit for the Purpose it is primarily used for (directory upload).

Our attempt at a better design is here: https://wicg.github.io/directory-upload/proposal.html

Again, if there needs to be a change, we'd prefer not to just unprefix this Entries API.

[triblondon]

Does directory upload cover entirely local-use use cases? Say for a photo editing app that wants to open local files, mutate them and save them back to the local file system. Neither spec seems to cover this use case. The defunct FileSystem API does, but was (still is?) only supported by Chrome: https://www.html5rocks.com/en/tutorials/file/filesystem/

[inexorabletash]

The FileSystem API dealt with two parts: (1) the file/directory/filesystem types, and (2) the ability to create a "sandboxed" file system for origin-specific storage. Chrome re-used the former for drag-and-drop of directories providing an "isolated" file system, and other browsers implemented the subset necessary for this. The doc that this issue references is a codification of that common subset.

You could imagine multiple sources of such filesystems, such as Chrome provides:

• a sandboxed "best effort" filesystem (akin to Indexed DB or the Cache API) - provided in Chrome via webkitRequestFileSystem(TEMPORARY, ...)
• a sandboxed filesystem with higher durability guarantees - provided in Chrome via webkitRequestFileSystem(PERSISTENT, ...)
• non-sandboxed "isolated" filesystem provided by certain operations (e.g. directory upload) - provided indirectly by Chrome via webkitGetAsEntry()
• non-sandboxed local/native file system access - with some security caveats, e.g. perhaps just for "installed" apps. Chrome has this for installed apps/extensions via chrome.* APIs. Not web-facing.
• non-sandboxed "sync" file system that is backed by cloud storage - Chrome has this again for apps/extensions via chrome.* APIs. Not web-facing.

It would be reasonable to assume that same file system API (file/directory entries, path resolution, directory traversal, access to file contents, etc) would be used for all of the above cases, as indeed Chrome does.

Despite recurring interest, no browser has yet tackled web-exposing direct native file/local file system access. Should that happen, we'll need to decide if that should re-use the types here or provide alternate types.

[inexorabletash]

It looks like most of @travisleithead's notes apply to the way that entries appear on the HTMLInputElement rather than the interfaces of the types that are exposed (and might be re-used for other FS-like operations).

1. The technique for getting entries from elements is synchronous (blocking).

^^^ Specific to HTMLInputElement

2. Poor ergonomics: there's a weird indirection to get a File from the Entries objects

^^^ Not specific to HTMLInputElement. However, this turns out to be useful; you may want to be able to persist a reference to a file distinct from the file itself. Consider an API which grants some permission for the local file system. The user opens "foo.png" for editing. The web app can persist the reference to the file distinct from the bits, so that when the web page is visited again the filename can be presented in e.g. File > Recently Used and the file re-opened.

3. Poor ergonomics: folder path information was added to the flat-list structure. This didn't mesh well with developer's intuitive method for an iterative or recursive approach to "traversing" directories.

^^^ Specific to HTMLInputElement. The rest of the API allows traversal.

4. Poor ergonomics: no use of Promises for async stuff :-(

^^^ As noted, these could be added to the types, if there was belief that these types were not dead ends.

5. The Entries API has "bolted on" access points that spread into a variety of other objects (e.g., has to extend File to provide *getEntries access points).

^^^ HTMLInputElement and DataTransferItem.

6. Empty folders weren't enumerated (didn’t provide an entry).

^^^ Specific to HTMLInputElement. Empty directories would be present using the traversal.

Our attempt at a better design is here: https://wicg.github.io/directory-upload/proposal.html

Yep. It's a cleaner API. Unfortunately, none of the browser vendors (including Microsoft) have yet elected to implement and ship it. :(

[inexorabletash]

To clarify: all of @travisleithead's feedback is a great reason to not just unprefix webkitdirectory.

A plausible incremental path forward building on what browsers have now:

• add a directory attribute to HTMLInputElement and expose an FrozenArray<FileSystemEntry> entries property which is populated on selection and contains only what was explicitly selected by user, with no extra traversal done synchronously. If a single file or multiple files are selected, this would be one or more FileSystemFileEntry. If a directory is selected it would be a single FileSystemDirectoryEntry, which developers would traverse. (This addresses points 1, 3, 5 and 6)
• add modern async APIs to the interfaces: Promise-based getters, and async iterator support for FileSystemDirectoryReader. (i.e. what the helper examples in the spec do) (This addresses point 4)

(Since Edge doesn't support the methods on the interfaces today this should result in a fairly clean interface in Edge. Other browsers would have both the callback APIs and Promise/async iterator APIs for back-compat.)

[annevk]

@inexorabletash can we please do #215 (comment) first? I'm sure we'll uncover more as we undertake that work.

[inexorabletash]

@annevk - I have no actual implementation or spec work planned here; all the above is just handwaving in response to questions/concerns.

Doing integration with HTML/FileAPI before anything else is fine with me. Not something I'm likely to spend time on in the near future, though.

[travisleithead]

It doesn't look we have any actionable outcomes here... looks like general consensus not to do anything here until a deeper integration with HTML is worked on and/or implementer interest changes. It seems unnecessary for us to keep this review open.