Markdown blocks have trouble rendering unicode emoji

[jameschensmith]

Summary

When using unicode emoji in a markdown block, the output appears to be getting trimmed. I tried to see if this was an issue with goldmark, but I don't think it is. I see that there is goldmark-emoji for :joy:-style emojis, but it didn't look like d2 was using this.

Issue appears to be for all layout engines.

Open to working on this if provided some guidance (would be my foray into the codebase).

Resources

D2 test file

Single emoji: |md
  🐵
|

Single emoji with text at beginning: |md
  This is a monkey emoji: 🐵
|

Single emoji with text at end: |md
  🐵 That was a monkey emoji.
|

Single emoji with text around: |md
  This is a monkey emoji: 🐵 It's pretty cool.
|

Multiple emojis: |md
  🐵🐵🐵🐵
|

Multiple emojis with text at beginning: |md
  These are monkey emojis: 🐵🐵🐵🐵
|

Multiple emojis with text at end: |md
  🐵🐵🐵🐵 Those were monkey emojis.
|

Multiple emojis with text around: |md
  These are monkey emojis: 🐵🐵🐵🐵 They are pretty cool.
|

— Playground Link

Exported SVG & PNG

[bo-ku-ra]

is the following relevant?
#517
#513

[alixander]

@jameschensmith if you're open to looking into this, I would look here to start:

d2/lib/textmeasure/markdown.go

Line 88 in 8fa00ca

func MeasureMarkdown(mdText string, ruler *Ruler, fontFamily *d2fonts.FontFamily) (width, height int, err error) {

For Markdown to work, D2 has to manually measure everything and match it with CSS. For example, H1 tags have x padding, y line height, z margin.

d2/lib/textmeasure/markdown.go

Line 33 in 8fa00ca

LineHeight_h = 1.25

The way fonts work, we measure each glyph:

d2/lib/textmeasure/textmeasure.go

Line 172 in 8fa00ca

func (t *Ruler) MeasurePrecise(font d2fonts.Font, s string) (width, height float64) {

If you can reproduce this with labels, then it's not a markdown problem. But that might not be ascertainable, since we pad labels, so it won't have that noticeable cutoff.

I suspect the emoji unicode gets measured incorrectly. I'm not really sure what the correct measurement should be, whether it's constant or some unit size of the font-size.

Good luck and please let me know if you get stuck, happy to dig into it more with you!

[jameschensmith]

Thanks, @alixander. Did a little more debugging this morning, and wanted to leave some more notes here. I did some more tests with other emojis, and the results are very strange. For instance, the cloud emoji (☁️) renders with proper width, as does a very complex emoji like "couple with heart: woman, man, medium-light skin tone" (👩🏼‍❤️‍👨🏼) which consists of multiple runes. Then you have the monkey emoji (along with others), and it gets cropped. Here's a simple playground showing two emojis measured differently. Also, I believe that this playground example shows that it may not be just markdown.

Basically, It doesn't look like it has to do with the amount of runes an emoji requires.

Unicode emoji resources:

• https://www.unicode.org/Public/emoji/15.0/emoji-sequences.txt
• https://www.unicode.org/Public/emoji/15.0/emoji-zwj-sequences.txt

[alixander]

oh interesting. maybe the font we use, Source Sans Pro, actually just has a subset of emojis supported.

adobe-fonts/source-sans#85

Maybe we need to package an emoji font like https://github.com/samuelngs/apple-emoji-linux, detect if the user used any emojis, and then inject that font into their SVG if so.

d2/d2renderers/d2svg/d2svg.go

Line 1062 in 0da4e15

for _, t := range triggers {

[jameschensmith]

Another update (things are getting clearer the more tests I run). The cloud emoji used in the last example is actually two runes. So basically, the emojis that are only one rune are the ones that get cut, whereas the emojis that are multiple runes can actually render too wide. I've provided an example below which shows emojis using 1-8 runes (ordered from smaller to larger). This may actually be a good regression test to add in the future.

Playground link

[alixander]

hah, wow interesting, til about emojis

[jameschensmith]

This is pretty complicated 😅 I've read in some places that emojis are double-width characters, but also that not all emojis have the same width (I think flags are one of those exceptions). I also read here that best solution might be to check the font for the glyph width. In any case, I think one improvement to what exists now would at least be to treat characters that consist of sequences of runes as a single glyph. Then, treating the emoji glyphs as double-width, or some other solution could come after that.

[jameschensmith]

If my understanding of the code is correct, atlas contains a mapping of ASCII codepoints (32 to 127 inclusive), and uses that mapping to draw the bounding box:

d2/lib/textmeasure/textmeasure.go

Lines 20 to 28 in 0da4e15

	// ASCII is a set of all ASCII runes. These runes are codepoints from 32 to 127 inclusive.
	var ASCII []rune
	func init() {
	ASCII = make([]rune, unicode.MaxASCII-32)
	for i := range ASCII {
	ASCII[i] = rune(32 + i)
	}
	}

And DrawRune replaces any non-ASCII codepoints with the unicode replacement character:

d2/lib/textmeasure/atlas.go

Lines 121 to 123 in 0da4e15

	if !a.contains(r) {
	r = unicode.ReplacementChar
	}

So all non-ASCII codepoints (including emojis) have the same bounding box as the unicode replacement character. What we could do is render any unsupported character as the unicode replacement character, until those other characters are supported. This sounds like a bad idea, I know 🙈 There's quite a bit of problems involved here, as I've noted in previous comments.

[alixander]

ohh that makes sense. I don't think the shim is necessary though, better to overflow into the padding a bit imo.

My fear is just that even if we measure this "right", if it's not part of the font we use, and users on different machines will just fallback to their OS's emoji font, which will over/underflow anyway. I don't think there's an option other than having the TTF support a large corpus of emojis

[alixander]

Might have to implement this #638 first. Emoji packs might be huge to embed just for a 🙈

[bo-ku-ra]

fyi. https://github.com/rivo/uniseg

[alixander]

@bo-ku-ra oh wow, good find. this looks incredibly relevant

[bo-ku-ra]

japanese uses double-byte characters.
so there seems to be a lot of information about it on japanese websites.