HTML entities in links are converted to symbols even if they do not end with a semi-colon

[jpurpleman]

Issue Overview

When we have content in the classic editor that is a web link with arguments. This link has query string arguments that include "&reg". In the classic editor the web link is fine, but when viewing the page in the Gutenberg editor, we see the "&reg" gets converted to a registered symbol.

Steps to Reproduce (for bugs)

Have a link with query arguments of "&reg"

Our simple example is

https://www.example.com/?foo=bar&reg=baz

1. Create a page in the classic editor
2. Put in the following content into text version of editor
3. https://www.example.com/?foo=bar&reg=baz
4. Save the page.
5. Edit the page in Gutenberg editor
6. You'll see that the "&reg" is now a registered symbol

Using Google Chrome on Linux Version 60.0.3112.90 (Official Build) (64-bit)
Using Gutenberg 2.4
Using WordPress 4.9.4

Expected Behavior

I don't expect the argument to change to a registered symbol

Screenshots / Video

Please see for before and after screenshots.

https://imgur.com/a/bNcb3

[ZebulanStanphill]

It looks like this happens with any string of characters that would be an HTML entity if you added a semicolon. (Not just in links.)

If you put &amp, &lt, &gt, or &#34 into a Gutenberg post using the Code Editor mode and then switch to the Visual Editor mode, the strings get converted to &, <, >, and " respectively.

I think the issue here is something along the lines of strings being parsed as HTML entities regardless of whether they end with a semicolon or not. Perhaps the strings are being considered as having typos and the semicolons are being added by automatic typo-correction?

[designsimply]

Tested and confirmed that HTML character entities that do not end with a ; are converted directly into symbols for inline URLs and converted from something like&cent to &cent inside an href attribute inside an anchor tag when opened in the Gutenberg editor.

this:

https://www.example.com/?foo=bar&reg=registered&amp=ampersand&quot=quote&cent=cent

<a href="https://www.example.com/?foo=bar&reg=registered&amp=ampersand&quot=quote&cent=cent">link</a>

&reg

&amp

&quot

&cent

becomes:

<p>https://www.example.com/?foo=bar®=registered&amp;=ampersand"=quote¢=cent</p>
<p><a href="https://www.example.com/?foo=bar&amp;reg=registered&amp;amp=ampersand&amp;quot=quote&amp;cent=cent">link</a></p>
<p>®</p>
<p>&amp;</p>
<p>"</p>
<p>¢</p>

^{Seen at http://alittletestblog.com/wp-admin/post.php?post=14002&action=edit running WordPress 4.9.7 and Gutenberg 3.2.0 using Firefox 61.0.1 on macOS 10.13.5.}

[davewarfel]

I noticed a similar issue today. Just wanted to document my experience and provide a screencast for further debugging.

Screencast:
https://www.dropbox.com/s/anom2rpsdtqfdvg/gutenberg-code-tags-html-visually.mp4?dl=0

Reproduce issue:

• Edit Visually: Enter some HTML code in a paragraph block (ex: <span class="">)
• Switch to Edit HTML: The < gets converted to < but the > stays a >
• Edit HTML: Wrap <code> tags around the HTML code, and change the > to a >
• Switch to Edit Visually: Looks fine
• Switch back to Edit HTML: The > is converted back to >

[mcsf]

@davewarfel: that is a different matter from the one reported in this issue. That discrepancy between < and > is actually the result of following the W3C specification. Gutenberg conforms to the spec in its serialization stack. See https://github.com/WordPress/gutenberg/blob/master/packages/escape-html/src/index.js#L70-L84.

@jpurpleman, I can't repro this issue. Most likely it was solved once we switched to our own serializer.

[LucCole]

This is still an issue.

When I try and add a double quote in my post with " it will convert it to a ".

If I try and add a &quot (no ;) then it will save fine the first time but when I go back to the editor it will convert it to a ".

This is a problem because,

Lets say I have a shortcode in my WP post

[my_shortcode text="This is some example text, and this is a "quote""]

Like I said this works fine if I can use

[my_shortcode text="This is some example text, and this is a &quotquote&quot"]

But the editor keeps changing (&quot) back to (")

[mcsf]

See #13609.

[adamjrice]

I'm also getting caught on this in version 5.2.3.

Also, interestingly, when I search posts from the admin interface for &, that gets converted to & -- which makes it that much harder to look for those escaped ampersands.