LineEdit can not input Chinese Characters on iOS devices

[alexzheng]

Godot version:

3.2.3

OS/device including version:

iOS 14.2

Issue description:

when try to input Chinese, nothing show up.

Steps to reproduce:

Minimal reproduction project:

[Calinou]

@alexzheng Did you change the font in your project to use a font that can display Chinese characters? The default project font only contains Latin-1 characters. Unknown characters will be invisible.

[alexzheng]

Yes, I have changed the font that can display any Chinese Character and it can works fine on Mac and Android device, when running on iOS device, It can show Chinese Character if I set it directly, but can not input by the soft keyboard.

[alexzheng]

This is the normal keyboard when input Chinese:

This is the keyboard show in Godot app That Chinese can not input

[alexzheng]

Also, the web export has the same issue.

[naithar]

This seems to be an issue with keyboard suggestions. Or maybe GodotView is missing some specific protocol compliance (like UITextInputTraits) that is required for suggestions view to show up.

But this doesn't really explain why web export has the same problem.

@alexzheng do you have the same problem with current Godot's 3.2 branch or 3.2.4 beta release? Can you reproduce it with older iOS versions, like 13 or 12?

[alexzheng]

godot 3.1.x 3.2.x
iOS 10.x iOS 12.x iOS 14.x all the same result.

[naithar]

Well, my guess was correct - to display keyboard's suggestion view, first responder have to implement UITextInput protocol instead of UIKeyInput.
But it seems pretty heavy to implement manually.
Making a backing UITextView to handle input seems to be a way.

[naithar]

@alexzheng any chance you could build and test this branch: naithar@74d5be7?
I've made a subclass from UITextView which handles keyboard display and fixed character input, using UITextViewDelegate instead of direct methods (Chinese characters wasn't handled by insertText or replaceText).
I've also tested it myself (with NotoSans font) and everything seems to work correctly, even dictation is working.

[alexzheng]

@naithar It works now.
It has some tiny problem, input some English Characters first then input some Chinese Characters, press the delete key on the soft keyboard, some English Characters can not be deleted.

[alexzheng]

another issue:
English Characters for the pinyin of Chinese also get input, this should not happen.

for example, to input the Chinese Character "我“, type "wo", then select "我“ from the suggestion view, only the Chinese "我" should be input, however, the "wo" is also input, it can not be deleted.

[alexzheng]

BTW, another issue related to keyboard is the the keyboard would popup unexpected.

kloder-games/godot-admob#94

[naithar]

BTW, another issue related to keyboard is the the keyboard would popup unexpected.

kloder-games/godot-admob#94

That's hardly related to this issue.

another issue:

English Characters for the pinyin of Chinese also get input, this should not happen.

for example, to input the Chinese Character "我“, type "wo", then select "我“ from the suggestion view, only the Chinese "我" should be input, however, the "wo" is also input, it can not be deleted.

So "wo" should be simply replaced with Chinese character?

[alexzheng]

Yes, admob Ads popup keyboard problem is not related to this issue, I just mention it since it seems like it is caused by the Godot Engine.

not replace, The "wo" should not be input, when the Chinese candidate Character "我" is chosen, only “我” should be displayed in the LineEdit, but now the issue is it displayed "wo我"， and the "wo" can not be deleted.

[Zireael07]

@naithar: Tip: check how IME is handled on iOS. In a properly working IME, "wo" should be replaced with the character.

[naithar]

Thanks for the info. I'll try to make it work correctly soon.

[naithar]

Yes, admob Ads popup keyboard problem is not related to this issue, I just mention it since it seems like it is caused by the Godot Engine.

I'm not really sure.
Godot displays keyboard only when show_keyboard method is called.
Can you reproduce this with my branch?
Making a symbolic link for becomeFirstResponder message should show a stack-trace for debug build, so it should be pretty easy to find thing that causes this issue.

[alexzheng]

It seems like your branch use ARC, so the admob module can not built.

It may not popup by the function show_virtual_keyboard since no output message is printed in this function in debug build.
void _show_keyboard(String p_existing) { keyboard_text = p_existing; printf("instance on show is %p\n", _instance); [_instance open_keyboard]; };

The scene do not contain any input Control such as LineEdit to TextEdit.

[naithar]

You can add a -fno-obj-arc compiler flag for specific module to build it with no ARC support.

And catching _show_keyboard will hardly give any result.

[alexzheng]

I'm not sure if this is the correct way to add this flag

the content of SCsub file of this module:

#!/usr/bin/env python

Import('env')
Import("env_modules")

env_admob = env_modules.Clone()

sources = [
'register_types.cpp',
'ios/src/admob.mm'
]

if (env["platform"] == "iphone"):
env_admob.add_source_files(env.modules_sources, sources)
env_admob.Append(CCFLAGS="-fno-objc-arc")

It still failed to compile, so sorry for I can not reproduce the issue in your branch.

[alexzheng]

For more info of this unexpected popup keyboard:
It can always reproduce by the following steps:
1 show a banner ad
2 click the ad, it will open another app
3 come back to the Godot app
4 wait the banner to refresh
5 the keyboard popup

Call hide_virtual_keyboard at the point of the banner refresh can stop the keyboard popup, Do this means the keyboard is triggered by the GLView of Godot?

if the banner is not clicked, no popup when refresh.

[Zireael07]

@alexzheng: This means it's most likely a problem with the admob addon.

[naithar]

I'm not sure if this is the correct way to add this flag

the content of SCsub file of this module:

#!/usr/bin/env python

Import('env')
Import("env_modules")

env_admob = env_modules.Clone()

sources = [
'register_types.cpp',
'ios/src/admob.mm'
]

if (env["platform"] == "iphone"):
env_admob.add_source_files(env.modules_sources, sources)
env_admob.Append(CCFLAGS="-fno-objc-arc")

It still failed to compile, so sorry for I can not reproduce the issue in your branch.

I've localy reverted ARKit to not use ARC and modified SCSub file to make it look like this:

#!/usr/bin/env python

Import("env")
Import("env_modules")

env_arkit = env_modules.Clone()

# (iOS) Enable module support
env_arkit.Append(CCFLAGS=["-fmodules", "-fcxx-modules"])
env_arkit.Append(CCFLAGS=["-fno-objc-arc"])

# (iOS) Build as separate static library
modules_sources = []
env_arkit.add_source_files(modules_sources, "*.cpp")
env_arkit.add_source_files(modules_sources, "*.mm")
mod_lib = env_modules.add_library("#bin/libgodot_arkit_module" + env["LIBSUFFIX"], modules_sources)

Everything is working fine, so I guess you should report it to admob maintainer or check compilation flags with verbose=yes scons option flag. -fno-objc-arc should be present, if compilation error is related to ARC.

Edit:

@alexzheng I've tried compiling master branch admob from https://github.com/kloder-games/godot-admob.
I've had to change [AppDelegate getViewController] to [AppDelegate viewController], add some additional header imports from platform code, -fno-objc-arch compilation flag and a path to GoogleMobileAds framework. Everything was built correctly after that with recent GoogleSDK (7.68.0).
I suggest you check error messages from compiler as ARC might not be related to you compilation failure.
I've published the code here: naithar@088b283, you'll just have to provide framework files. I haven't fixed the replacement issue yet, but it should probably allow you to test admob's mater branch.

I would also suggest you try to debug an application with symbolic breakpoints so you can catch methods like becomeFirstResponder which is responsible for keyboard display.
Other that that it's hardly a Godot's issue and should be fixed in module itself, which seems to be the case with admob's fork at poingstudios/godot-admob-android@ff72edc.

And we should probably stick to issue from OP in this discussion :)

[alexzheng]

This fix is a workaround, it just call [rootController.view endEditing:YES] to resigns first responder when the keyboard is about to popup. I have already did something like this to ignore this issue. The problem is when the ads is loaded, it should not cause the rootViewController popup keyboard, it is not required to hide the keyboard when ad is refreshing from the admob document.

OK, too much discussion here, let's focus on the Chinese input issue

[alexzheng]

A good news is it seems the keyboard popup issue have been fixed in your branch :)

[naithar]

@alexzheng I've made some changes to my branch: naithar@890be81
Everything seems to work as expected, but double checking wouldn't hurt.

A good news is it seems the keyboard popup issue have been fixed in your branch :)

That's good. But Godot changes might not be related, since admob module was updated too.

[alexzheng]

That's good. But Godot changes might not be related, since admob module was updated too.

That's related to your changes, I did not update my local admob module. Although the change in the admob module can just hide the keyboard.

[naithar]

That's good. But Godot changes might not be related, since admob module was updated too.

That's related to your changes, I did not update my local admob module. Although the change in the admob module can just hide the keyboard.

It could also be fixed by more earlier changes made for 3.2.4 release, but if it's working it's fine either way :)

Let me know when you check this changes. I'll port them to 4.0 and make a PR.

[alexzheng]

It's too slow to download the zip file for the branch, did you only Changed the keyboard_input_view.mm compared to your first commit?

I have test it by only update this file from this naithar/godot@74d5be7, it still have some problem.

Can you try to type feichang and select 非常, then ganxieni and select 感谢你，the text should be 非常感谢你， but now is 感谢你

Only the pinyin for the Chinese should be replaced, the previously entered Chinese should not :)

[naithar]

Only the pinyin for the Chinese should be replaced, the previously entered Chinese should not :)

The reason for this is that for some reason iOS sometimes make some symbols count as 2 characters instead of one:

insert at: {0, 0}, 'f'
insert at: {1, 0}, 'e'
insert at: {2, 0}, 'i'
insert at: {3, 0}, 'c'
insert at: {5, 0}, 'h'
insert at: {6, 0}, 'a'
insert at: {7, 0}, 'n'
insert at: {8, 0}, 'g'
replace at: {0, 9}, 非常
insert at: {2, 0}, 'g'
insert at: {3, 0}, 'a'
insert at: {4, 0}, 'n'
insert at: {5, 0}, 'x'
insert at: {7, 0}, 'i'
insert at: {8, 0}, 'e'
insert at: {9, 0}, 'n'
insert at: {11, 0}, 'i'
replace at: {2, 10}, '感谢你'

[alexzheng]

Can it behavior like the native textView?

[naithar]

It does behave like native textView, since it's a native iOS UITextView.

[naithar]

insert at: {0, 0}, 'f'
insert at: {1, 0}, 'e'
insert at: {2, 0}, 'i'
insert at: {3, 0}, 'c'
insert at: {5, 0}, 'h'
insert at: {6, 0}, 'a'
insert at: {7, 0}, 'n'
insert at: {8, 0}, 'g'
replace at: {0, 9}, '非常', string to replace: 'fei chang'
insert at: {2, 0}, 'g'
insert at: {3, 0}, 'a'
insert at: {4, 0}, 'n'
insert at: {5, 0}, 'x'
insert at: {7, 0}, 'i'
insert at: {8, 0}, 'e'
insert at: {9, 0}, 'n'
insert at: {11, 0}, 'i'
replace at: {2, 10}, '感谢你', string to replace: 'gan xie ni'

Seems like UITextView delegate does not notify about space insertion if it's done based on language.

[alexzheng]

Yes, the space between each pinyin for a Chinese Character is not reported, but we can notice it from the location of range, the textView in native app will insert a space automatically, and if you press the "确认" to accept the English, the space will be removed. That's a little complicated.

[naithar]

naithar@05048bf this should fix some of the incorrect behavior, but I'm not really happy with this implementation.

[alexzheng]

Yes, this is only a special behavior for Chinese Input, some other non-ascii languages may have their own special behavior, such as Japanese or Korean, I do not know these languages.

[naithar]

@akien-mga any chance someone could explain how android handles keyboard input?
From what I understand from GodotTextInputWrapper it removes all text and enters it again on change notification. I can't test it, but I don't understand how TextEdit cursor is handled if it's handled.

[naithar]

Another common issue on all platform that the LineEdit is different from a usual Edit is that text will not fulfill the area of the control on delete.

@alexzheng you should probably create another issue with MRP for this :)

[akien-mga]

any chance someone could explain how android handles keyboard input?
From what I understand from GodotTextInputWrapper it removes all text and enters it again on change notification. I can't test it, but I don't understand how TextEdit cursor is handled if it's handled.

I don't know if anyone is currently familiar with that code, but maybe @m4gr3d @pouleyKetchoupp or @RandomShaper have insights into how it works.

As for IME for CJK specifically, @volzhs and @bruvzg both did work on this area, but I don't know if that's transferable to help with iOS specifically.

[alexzheng]

OK, created #43438

[naithar]

@alexzheng have you tested naithar@05048bf branch? Is it working okay with Chinese keyboard now or it still have some issues?

Edit:

naithar@a0ad5cd made some more changes to the branch, hopefully if it's working correctly I can start refactoring it.

[alexzheng]

@naithar
Still have some other issue,
I just types some key randomly, for example type ni, select 你， type nimwnm, and press "确认" ， the text is 你ninimwnm, it should be 你nimwnm

There may many other traps in input, so it's may not a good idea to tweak the implementation for each specific behavior.

What about trying to get text directly from the text property of the native textView? it has done all the tricks for each input method.

[naithar]

What about trying to get text directly from the text property of the native textView? it has done all the tricks for each input method.

Godot doesn't report caret/cursor position (cursor_start and cursor_end is -1), so there is now way to correctly transfer the text without this information. So it would be helpful if someone from Android could give some advice on this.

Edit:

This seems to be the case with TextEdit, but LineEdit seems to be reporting cursor data.

[naithar]

@alexzheng naithar@de73e0a this should do it, I guess. I've modified TextEdit to send a cursor data in notification. So now native text input should be able to correctly translate it's own input into Godot's.
Unicode characters like Six-Per-Em Space that's not handled by font or Godot might still be a problem, but we wouldn't find them without more testing.

Since I've modified TextEdit to send additional data (cursor_start, cursor_end) it might be a good idea to test how Android handles new TextEdit changes (LineEdit wasn't modified, so it would work the same), but I can't really test it.