Make FontData importable resource. Add multi-channel SDF font rendering.

[bruvzg]

Make FontData importable resource.
Add multi-channel(+ true) SDF font texture generation and rendering support.
Add FontData import plugins, font texture cache pre-generation and loading.

Supersede #44772

Screen.Recording.2021-08-23.at.18.40.11.mov

• M(T)SDF is updated to v1.9.1, and modified to be multithreaded (using Godot ThreadWorkPool instead of OpenMP), it's still too slow to use without pre-rendering.
• No custom file formats, FontData is stored as normal resource.
• It's now possible to make custom importers for other bitmap font formats, or construct FontData from scripts.
• Included importers (all import to the same FontData resource):
  • Dynamic font importer (with MSDF support).
  • BMFont bitmap font importer.
  • Image to font (included as the example for other font type importers).

Sample of generated M(T)SDF texture (source size = 40, pixel range = 8):

[Calinou]

I would probably rename the image font importer to clearly denote that it supports fixed-width fonts only. For instance, this could be done by changing its visible name to Font Data (Image, Fixed Width Only).

[fire]

Using a rich text node and a font corrupts the tscn on load.

The rich text is below.

Godot[a] (/ɡəˈdoʊ/ ) is a cross-platform, free and open-source game engine released under the MIT license. It was initially developed by Argentine software developers Juan Linietsky and Ariel Manzur[5] for several companies in Latin America prior to its public release.[6] The development environment runs on multiple operating systems including Linux, BSDs, macOS, and Microsoft Windows. It is designed to create both 2D and 3D games targeting PC, mobile, and web platforms. 

Godotは、クロスプラットフォームかつオープンソース（MITライセンス）の2D/3Dゲームエンジンである。現在の開発主体はGodot Engine communityであるが、オープンソース化によってそのような形態をとる前はラテンアメリカの幾つかの企業向けに開発、使用がなされていた[4]。開発環境（エディタ）はWindows、macOS、Linux、BSD系OS とHaiku（32ビット及び64ビット）で動作し、開発対象となるプラットフォームはPC、家庭用ゲーム機、携帯電話、Web (HTML5、WebGL) と、非常に多様なプラットフォーム上で動作するように設計されている。 

[bruvzg]

Update:

• Fixed multiple multi-threading related issues.
• Added a second page to the export dialog to preload glyphs based on the entered text. Some presentation forms (e.g. shaped Arabic text, OpenType stylistic alternatives) do not have character associated with them and can't be selected from the character map (which includes only base characters).

Demo (includes imported font data):
test_font_import_preload.zip

[bruvzg]

Note: overlapping outlines is an unrelated regression, and is fixed by #52012

[lyuma]

The main crash I was hitting is fixed now.! I took a .ttf and tried messing with the import settings to see what would happen but I ran into a crash.

Steps to reproduce:
I have a .ttf, go to the Import dock. All the PackedStringArray's are size 0.
I didn't go to Advanced... Instead, I expanded Enabled Languages directly from the import dock, add an item, type "en", then I clicked Reimport. After doing this, godot crashes.

I understand this probably wasn't how it was intended to be used, but I didn't know how to use it so I assumed "en" was the name of a language.

ERROR: FATAL: Index p_index = 2 is out of bounds (size() = 2).
   at: CowData<class String>::get (S:\repo\godot-pr-pr\core/templates/cowdata.h:157)
CrashHandlerException: Program crashed
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[0] CowData<String>::get (S:\repo\godot-pr-pr\core\templates\cowdata.h:157)
[1] Vector<String>::operator[] (S:\repo\godot-pr-pr\core\templates\vector.h:89)
[2] FontData::_get (S:\repo\godot-pr-pr\scene\resources\font.cpp:311)
[3] FontData::_getv (S:\repo\godot-pr-pr\scene\resources\font.h:43)
[4] Object::get (S:\repo\godot-pr-pr\core\object\object.cpp:514)
[5] ResourceFormatSaverBinaryInstance::_find_resources (S:\repo\godot-pr-pr\core\io\resource_format_binary.cpp:1766)
[6] ResourceFormatSaverBinaryInstance::save (S:\repo\godot-pr-pr\core\io\resource_format_binary.cpp:1875)
[7] ResourceFormatSaverBinary::save (S:\repo\godot-pr-pr\core\io\resource_format_binary.cpp:2063)
[8] ResourceSaver::save (S:\repo\godot-pr-pr\core\io\resource_saver.cpp:110)
[9] ResourceImporterDynamicFont::import (S:\repo\godot-pr-pr\editor\import\resource_importer_dynamicfont.cpp:289)
[10] EditorFileSystem::_reimport_file (S:\repo\godot-pr-pr\editor\editor_file_system.cpp:1849)
[11] EditorFileSystem::reimport_files (S:\repo\godot-pr-pr\editor\editor_file_system.cpp:2066)
[12] ImportDock::_reimport (S:\repo\godot-pr-pr\editor\import_dock.cpp:529)
[13] ImportDock::_reimport_attempt (S:\repo\godot-pr-pr\editor\import_dock.cpp:462)
[14] call_with_variant_args_helper<ImportDock> (S:\repo\godot-pr-pr\core\variant\binder_common.h:227)
[15] call_with_variant_args<ImportDock> (S:\repo\godot-pr-pr\core\variant\binder_common.h:337)
[16] CallableCustomMethodPointer<ImportDock>::call (S:\repo\godot-pr-pr\core\object\callable_method_pointer.h:97)
[17] Callable::call (S:\repo\godot-pr-pr\core\variant\callable.cpp:51)
[18] Object::emit_signal (S:\repo\godot-pr-pr\core\object\object.cpp:1100)
[19] Object::emit_signal (S:\repo\godot-pr-pr\core\object\object.cpp:1155)
[20] BaseButton::_pressed (S:\repo\godot-pr-pr\scene\gui\base_button.cpp:126)
[21] BaseButton::on_action_event (S:\repo\godot-pr-pr\scene\gui\base_button.cpp:168)
[22] BaseButton::gui_input (S:\repo\godot-pr-pr\scene\gui\base_button.cpp:67)
[23] Control::_call_gui_input (S:\repo\godot-pr-pr\scene\gui\control.cpp:793)
[24] Viewport::_gui_call_input (S:\repo\godot-pr-pr\scene\main\viewport.cpp:1241)
[25] Viewport::_gui_input_event (S:\repo\godot-pr-pr\scene\main\viewport.cpp:1559)
[26] Viewport::push_input (S:\repo\godot-pr-pr\scene\main\viewport.cpp:2683)
[27] Window::_window_input (S:\repo\godot-pr-pr\scene\main\window.cpp:920)
[28] call_with_variant_args_helper<Window,Ref<InputEvent> const &,0> (S:\repo\godot-pr-pr\core\variant\binder_common.h:222)
[29] call_with_variant_args<Window,Ref<InputEvent> const &> (S:\repo\godot-pr-pr\core\variant\binder_common.h:337)
[30] CallableCustomMethodPointer<Window,Ref<InputEvent> const &>::call (S:\repo\godot-pr-pr\core\object\callable_method_pointer.h:97)
[31] Callable::call (S:\repo\godot-pr-pr\core\variant\callable.cpp:51)
[32] DisplayServerWindows::_dispatch_input_event (S:\repo\godot-pr-pr\platform\windows\display_server_windows.cpp:1803)
[33] DisplayServerWindows::_dispatch_input_events (S:\repo\godot-pr-pr\platform\windows\display_server_windows.cpp:1777)
[34] Input::_parse_input_event_impl (S:\repo\godot-pr-pr\core\input\input.cpp:623)
[35] Input::flush_buffered_events (S:\repo\godot-pr-pr\core\input\input.cpp:843)
[36] DisplayServerWindows::process_events (S:\repo\godot-pr-pr\platform\windows\display_server_windows.cpp:1530)
[37] OS_Windows::run (S:\repo\godot-pr-pr\platform\windows\os_windows.cpp:626)
[38] widechar_main (S:\repo\godot-pr-pr\platform\windows\godot_windows.cpp:163)
[39] _main (S:\repo\godot-pr-pr\platform\windows\godot_windows.cpp:185)
[40] main (S:\repo\godot-pr-pr\platform\windows\godot_windows.cpp:199)
[41] __scrt_common_main_seh (D:\agent\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288)
[42] BaseThreadInitThunk
-- END OF BACKTRACE --

I also tried Advanced... but the languages section doesn't seem to work (I can add languages, checked or unchecked, but after clicking Reimport they get forgotten)

ERROR: Error calling from signal 'text_changed' to callable: DynamicFontImportSettings::_change_ftr : Method expected 0 arguments, but called with 1..
   at: Object::emit_signal (core\object\object.cpp:1111)
ERROR: Error calling from signal 'text_changed' to callable: DynamicFontImportSettings::_change_ftr : Method expected 0 arguments, but called with 1..
   at: Object::emit_signal (core\object\object.cpp:1111)
ERROR: Parameter "vars_item" is null.
   at: DynamicFontImportSettings::_re_import (editor\import\dynamicfont_import_settings.cpp:1116)
ERROR: Parameter "vars_item" is null.
   at: DynamicFontImportSettings::_re_import (editor\import\dynamicfont_import_settings.cpp:1116)
ERROR: Parameter "vars_item" is null.

Suffice it to say that the import UI probably needs a bit of work. but that needs to be a separate PR. I'd love to see this one get merged.

[bruvzg]

I didn't go to Advanced... Instead, I expanded Enabled Languages directly from the import dock, add an item, type "en", then I clicked Reimport. After doing this, godot crashes.

Should be fixed now.

I understand this probably wasn't how it was intended to be used, but I didn't know how to use it so I assumed "en" was the name of a language.

It is intended way to use it. Use cases for this are automatic font selection for Chinese, Japanese, and Korean (same character set, but some looks a bit different) and selection of different Arabic script variants based on locale. Script override can be useful to disable a partially supported portion of the font (e.g. many fonts report Greek support, but only have a limited set of characters for mathematical notation).

I also tried Advanced... but the languages section doesn't seem to work (I can add languages, checked or unchecked, but after clicking Reimport they get forgotten)

Also fixed.

Suffice it to say that the import UI probably needs a bit of work.

It definitely does, and probably deserve a separate proposal to discuss how it should look.
So far, its typical usage look like this (which is probably not the most intuitive):

[bruvzg]

Update: Added some improvements to the BMFont importer:

• Binary format support (version 3) - https://www.angelcode.com/products/bmfont/doc/file_format.html
• Support for outlines.
• Support for different texture layouts - https://www.angelcode.com/products/bmfont/doc/export_options.html

Test project: bmfont_test.zip (include BMFont source config files).

[fire]

The msdf feature is great from a design standpoint.

The last concern was from reduz about the non-resource binary but that was resolved.

This pr is getting bigger, is it possible merge this and add future work in new prs?

@reduz

[fire]

Is the year correct?

[bruvzg]

As it is in the upstream license - https://github.com/Chlumsky/msdfgen/blob/d07fa1d2c8088f87463182a05cab9bee29844e70/LICENSE.txt#L3

[fire]

Thanks!

[jordo]

amazing work. thx so much for this!

[DrCanvas]

Hi, Is this a bug?

[Calinou]

@DrCanvas It's surely a bug. Could you create a new issue and upload a minimal reproduction project that includes the font? If the font's license does not allow redistribution, you can try reproducing this with another font.

[fire]

Can you double or triple your MSDF pixel range?

[bruvzg]

It's most likely an issue with the specific font. msdfgen is not good with handling self intersecting shapes when it's built without Skia (Skia is huge, and probably won't be integrated into Godot unless it's used for much more stuff than just MSDF generation), see Chlumsky/msdfgen#110.

[Geometror]

I noticed a regression with RTL text using specific fonts which was introduced in this PR:

(the used font is Cairo-Regular; with Tajawal-Bold there is no issue)

[bruvzg]

I noticed a regression with RTL text using specific fonts which was introduced in this PR

Should be fixed by #52292