Scope of our infoset

[HadrienGardeur]

In a recent email to the mailing list, @TzviyaSiegman said:

Ensure that most of the features and capabilities of EPUB are incorporated into WP/PWP/EPUB4. This is not the same thing as backward compatibility, nor does it mean that we are duplicating EPUB on the Web. It will be functionally compatible.

This was also discussed during our April 16th conference call.

Our current infoset was not designed to be aligned with EPUB 3 and has a much more limited subset of items. Are we backpedaling and re-aligning with the spirit of EPUB BFF (Browser Friendly Format)?

With EPUB BFF (and its descendent, the Readium Web Publication Manifest), while the format is not itself backward compatible, it's meant to be "round trippable" and cover the same infoset.

I can work on a gap analysis between the WP infoset and EPUB 3.2, but before that, I'd like to make sure that this is the new direction we're taking.

[GarthConboy]

I view such round tripping as important – even from out charter "This specification should generally be a functional superset of EPUB 3.1. Functional round-tripping to/from EPUB 3.1 considered highly desirable." (s/3.1/3.2).

I'd think @laudrain is looking at this as part of his infoset work?

[HadrienGardeur]

I view such round tripping as important – even from out charter "This specification should generally be a functional superset of EPUB 3.1. Functional round-tripping to/from EPUB 3.1 considered highly desirable."

Right, but that's currently listed under EPUB 4, not WP/PWP. Are we adopting this line of thinking for these two specs as well?

[GarthConboy]

I hope so!

[HadrienGardeur]

OK, well I'll do a gap analysis and comment here, but IMO this is a very serious reason for this group to consider the Readium Web Publication Manifest for its serialization.

RWPM was built to support the EPUB infoset, and while we don't do full round trips with it, it's used in all Readium-2 based products to expose EPUB over HTTP.

[dauwhe]

Perhaps not absolutely everything in every EPUB-related spec ever should be included in the WP infoset.

[HadrienGardeur]

I think we can easily make a difference between the core spec and the rest.

[WSchindler]

As the nature of the contents in a WP may widely differ, I agree that we will need to define a core infoset that covers the common ground for many publications. Nevertheless we should have a well-defined extension mechanism to be able to open up WP for important reference books such as dictionaries, encyclopedia, etc.

[iherman]

To put it another way what @WSchindler just said: we may have to "layer" or "prioritize" the Infoset items. We can decide to define some sort of a version 1 now, with the possibility to extend the infoset later.

[HadrienGardeur]

Here's a first take on this gap analysis, based on the EPUB 3.2 draft.

Metadata

• the EPUB infoset contains a link element that's currently missing from the WP infoset (this is tied to Linking to alternate representations #159, Linking to an external metadata record #162 and Linking to a privacy policy #163 as well)
• no explicit support for a list of rel either (in EPUB there's a dedicated document for that)
• there are no equivalents for dc:coverage, dc:description, dc:format, dc:publisher, dc:relation, dc:rights, dc:source, dc:subject or dc:type in WP
• there's no concept of a sortable string on certain metadata elements (opf:file-as) in WP
• it's not clear if our infoset allows for more than one title element (our WebIDL is currently limited to a single title) and if these titles could be sequenced or typed
• we don't have a generic metadata element (meta) or explicit support for external vocabularies
• no equivalent for some of our vocabularies such as the package rendering vocabulary, accessibility vocabulary, media overlay vocabulary or the more generic meta properties vocabulary
• for authors/contributors, we don't know yet how we'll handle the equivalent of opf:role (see Handling roles for creators #167)

Default Reading Order & List of Resources

• the EPUB infoset contains properties that are applied to the manifest (list of resources) and the spine (default reading order), none of them exist in our WP infoset
• for the spine, the properties are divided between package rendering and the rest, but they're all about spreads, orientation or the nature of the layout (reflowable vs fixed)
• for the manifest, there are a lot more properties:
  • some of them are meant to indicate what's present in the resource (mathml, svg, scripted, remote-resources)
  • while a few properties behave exactly like rel values (cover-image, nav) and indicate the relationship between the resource and the publication
• the WP infoset doesn't have the equivalent of media overlays either, where each resource in the default reading order could be tied to an audio resource

Navigation

• the WP infoset is extremely vague about table of contents and their implementation
• in the case of EPUB 3.2, what's possible with the Navigation Document goes beyond a table of contents, it's also possible to provide the following lists: page list, landmarks, list of audio files, list of tables, list of videos and list of images.

Packaging

• compared to EPUB 3.2, we don't have any idea yet how packaging will be handled
• but what falls under packaging in EPUB could be handled elsewhere in WP: that's the case for instance of encryption.xml and signature.xml, since we'll need encryption and signatures on the Web as well

[TzviyaSiegman]

Thanks @HadrienGardeur. This will be useful to @laudrain and the whole group as we proceed. We need to make sure that we keep a few principles in mind as we work on this.

1. We are looking to duplicate the functionality of EPUB and not necessarily the technical wiring of EPUB. For example, we may decide that externalized metadata is necessary, but we have not discussed whether the preferred method is link.
2. We are specifically not talking about packaging in WP. This will be addressed in PWP.
3. We need to assess what to do with the EPUB satellite specs, such as multiple renditions.

[HadrienGardeur]

We are looking to duplicate the functionality of EPUB and not necessarily the technical wiring of EPUB. For example, we may decide that externalized metadata is necessary, but we have not discussed whether the preferred method is link.

This has been my focus as well in this gap analysis. But for links, we lack "publication-level linking" as an infoset item, this is not tied to the technical wiring of EPUB.
This is not strictly tied to external metadata in EPUB or in WP.

We are specifically not talking about packaging in WP. This will be addressed in PWP.

That's why the section is very short, but we can hardly say that encryption and signatures are limited to packaging use cases.

We need to assess what to do with the EPUB satellite specs, such as multiple renditions.

I've mostly avoided them in this gap analysis.

[laudrain]

Thanks @HadrienGardeur !
I'm a bit concerned to compare EPUB 3.2 and WP. IMO most of the items you have listed in this gap analysis don't belong to WP.
But this analysis will be very useful when coming to EPUB4.

About links:

But for links, we lack "publication-level linking" as an infoset item, this is not tied to the technical wiring of EPUB.
This is not strictly tied to external metadata in EPUB or in WP.

Isn't addressed is 3.5 Extensibility point 2 ?

by the provision of linked metadata records.

[HadrienGardeur]

@laudrain

My goal wasn't really to assess what does or does not belong in a WP.

Since our latest discussions about the infoset indicate a shift to a more BFF-like approach, I'm simply evaluating the differences between the current EPUB 3.2 and WP infosets.

Isn't addressed is 3.5 Extensibility point 2 ?

I don't think that's enough. We need to know how those links are provided as well:

• in the manifest itself or its HTTP headers?
• in a resource from the publication or its HTTP headers?

Purely from a lifecycle perspective, I don't have a single clue how we're supposed to discover this information right now.

A separate infoset item for publication-level links would be the first step.

[GarthConboy]

Thanks @HadrienGardeur – nice work. @laudrain raises the key question that we'll need to address going forward, presuming we want to address many of these gaps, at which level should that be done (WP, PWP/EPUB4)? No single answer, I'm sure.

[HadrienGardeur]

The big one is FXL, it's by far where we have the largest gap between WP and EPUB 3.2.

It mostly affects spine properties (20+ items) and metadata (5+ items).

[dauwhe]

I was an editor of the FXL spec, and FXL metadata should not be part of web publications. Fixed-layout is at heart a CSS technique, and should be left to CSS. Metadata outside of CSS should not affect the actual rendering of web content.

For example, we should not be in the business of defining how two different content documents should appear in the same viewport. If content authors want to do something like that, they're welcome to use iframes and the appropriate CSS.

[HadrienGardeur]

Overall I agree that FXL should not be part of WP, but I think that's a very different debate for EPUB 4 if we want to be "infoset compatible" with EPUB 3.2

For example, we should not be in the business of defining how two different content documents should appear in the same viewport. If content authors want to do something like that, they're welcome to use iframes and the appropriate CSS.

The result would be fairly catastrophic in dedicated apps if you completely drop the ability to say that a given resource is pre-paginated.
This would also make things difficult for browsers who want to provide an option for switching between one page and two pages spreads (ping @BCWalters since Edge has that option).

[BCWalters]

I also generally agree that we should avoid FXL in WP but we'd need something explicit if we want to have any semblance of roundtripping between EPUB3 FXL and EPUB4.

If a content author's goal is to create a coherent two page spread in a multi-page paginated reading mode like in children's books, I don't see how iframes and CSS can help. Even if you have CSS properties to ensure that each iframe takes up one full logical page, you need to override the user agents ability to change fonts and colors, and you need the ability to force one to appear at the beginning of the visible viewport when multiple pages are visible. This seems like a tough sell for CSS, as it's going to be very difficult for user agents to support CSS properties relative to a higher level WP viewport.

[HadrienGardeur]

Based on the gap analysis, I've created a number of issues:

• Reference resources that are not part of the publication #186
• Bring back popular EPUB 2.x/3.x metadata? #187
• Sortable metadata strings #188
• Letting the author control pagination vs scrolling #207
• #208
• #209
• Hints about resources in "default reading order" or "list of resources" #222
• Other navigation elements beyond ToC #223

Once we've addressed some of these issues, I'll take another look at the draft for EPUB 3.2 and see if I can identify additional gaps.

[wareid]

As discussed on Feb 4 2019, closing this issue in favour of the related issues.