Impossible to use symbols as a comment marker

[SorielXD]

Tested versions

• 4.3.stable

System information

Godot v4.3.stable - Windows 10.0.22631 - GLES3 (Compatibility) - Intel(R) Iris(R) Xe Graphics (Intel Corporation; 31.0.101.5186) - 12th Gen Intel(R) Core(TM) i7-1255U (12 Threads)

Issue description

It seems the text editor does not support, symbols (such as : §, ?, !, !!) as comment marker

The comment doesn't use the corresponding color and keep the normal comment color compared to a comment marker with letters and numbers

Steps to reproduce

1. Open the customisation parameters of the text editor (Editor Settings -> Text Editor -> Theme)
2. Set the theme to Custom
3. Add symbols to the list (there are at the end)
4. Comment with these symbols

Minimal reproduction project (MRP)

N/A

[bruvzg]

Currently, markers names follow the same rules as GDScript identifiers: should starts with XID_Start class character or "_", and may contain only XID_Continue class characters.

[SorielXD]

@bruvzg I see, but then i think it should be mention in the description (the actual description is blank) or have some kind of warning because it feel just weird like this.

[dalexeev]

I can confirm that during the work on #79761 this limitation was intended. It is done to speed up comment marker searches using a hashmap. Yes, the limitation should be documented, we just need to move EDITOR_DEFs so the DocTools system can detect the setting.

should starts with XID_Start class character or "_", and may contain only XID_Continue class characters.

No, it only uses is_unicode_identifier_continue(). I'm not sure how important this is from a Unicode standards perspective.

godot/modules/gdscript/editor/gdscript_highlighter.cpp

Lines 205 to 231 in 8120e03

	if (color_regions[in_region].is_comment) {
	int marker_start_pos = from;
	int marker_len = 0;
	while (from <= line_length) {
	if (from < line_length && is_unicode_identifier_continue(str[from])) {
	marker_len++;
	} else {
	if (marker_len > 0) {
	HashMap<String, CommentMarkerLevel>::ConstIterator E = comment_markers.find(str.substr(marker_start_pos, marker_len));
	if (E) {
	Dictionary marker_highlighter_info;
	marker_highlighter_info["color"] = comment_marker_colors[E->value];
	color_map[marker_start_pos] = marker_highlighter_info;
	Dictionary marker_continue_highlighter_info;
	marker_continue_highlighter_info["color"] = region_color;
	color_map[from] = marker_continue_highlighter_info;
	}
	}
	marker_start_pos = from + 1;
	marker_len = 0;
	}
	from++;
	}
	from = line_length - 1;
	j = from;
	} else {