Adding SendKey to Terminal

[A-J-Bauer]

For custom HTML on screen keyboards and as a first step away from deprecated keyPress and hard key codes.
(For built in touch-device on-screen-keyboards a beforeinput event handler could call this, for desktop a keydown event handler)

[Tyriar]

I'm not really understanding this change, is there an issue tracking the problem? Also, it seems to be duplicating a lot of what happens in Keyboard.ts which causes bloat/tech debt and it's handling some keys differently:

https://github.com/Tyriar/xterm.js/blob/82f8e06ff683ebcac305ac14cd40321f358e304f/src/common/input/Keyboard.ts#L245-L252

[A-J-Bauer]

No there is no exact issue tracking the problem except partially #1862.
Reason behind this is the lack of a function where you can send keystrokes to the Terminal without sending fake KeyboardEvents or calling _core functions after a lot of digging. I need the functionality for a custom on screen keyboard.

Keypress event (actually only space handled there?) and hard keyCodes are deprecated and will break things in the future.
Well they already break things because of different keyboard layouts (~, [] a.s.o).
So I decided to find the predefined Key Values that can replace the keyCodes used in evaluateKeyboardEvent (assuming that it works on a US keyboard) and make an extra function using only predefined keys. This IMHO is a better approach than to mess up evaluateKeyboardEvent right away. If this function works as expected and after fixing potential issues it could be used in a keyDown event handler, composition handlers and beforeInput (with cancel).

In fact those handlers could be implemented and tested with regular javascript beforehand this way.

KeyPress https://developer.mozilla.org/en-US/docs/Web/API/Document/keypress_event
KeyCode https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/keyCode
Predefined keys https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/key/Key_Values

Thought maybe someone can use this,

Regards
A.J.

[Tyriar]

Thanks for the context. This sort of API could be valuable as I know some mobile terminals have shortcuts for ctrl+c/tab for example. I think the better approach would adapting the existing Keyboard.ts to work on key and then exposing the new sendKey API to just pass through to that.

Also, could the major case here be handled by input events? We started listening to that event recently to support emoji IMEs.

[A-J-Bauer]

"..adapting the existing Keyboard.ts to work on key.."
I think this is not easily done without breaking things in the process.

With a SendKey function on the other hand all this could be tried out using vanilla javascript with zero installation overhead. All thats needed is a second test input/textarea plus the dom events to get the keys (String) and then a call to SendKey. If this would prove to be working it could be moved if not it could be taken out again.

As far as my research goes handling events from device on screen keyboards should be done in beforeinput.

beforeinput https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/beforeinput_event

pseudocode:

textarea.addEventListener('beforeinput', function (e) {
        console.dir(e);
        if (e.inputType === 'insertText'
         || e.inputType === 'insertFromPaste') {
            //e.preventDefault();
            //e.stopPropagation();
            //e.stopImmediatePropagation();
            sendKey(e.data);                
        }
        else if (e.inputType === 'insertCompositionText') {
            e.preventDefault();
            //e.stopPropagation();
            //e.stopImmediatePropagation();
        }
    });`

[Tyriar]

The overhead in this PR is the duplicated code which increases the size of the library. I still understand the difficulty in adapting this function to move away from keyCode, don't we just need to map the numbers to the key strings and that's it? We also have some tests in Keyboard.test.ts to help with the migration:

xterm.js/src/common/input/Keyboard.ts

Lines 38 to 43 in 82f8e06

	export function evaluateKeyboardEvent(
	ev: IKeyboardEvent,
	applicationCursorMode: boolean,
	isMac: boolean,
	macOptionIsMeta: boolean
	): IKeyboardResult {

beforeinput sounds dangerous and will probably introduce other regressions (composition, IMEs, different keyboard layouts, etc.), I guess we should leave this up to embedders as you originally suggested with this new API.

[A-J-Bauer]

beforeinput is like input but before editable content changes, since you are using input..

https://bugs.chromium.org/p/chromium/issues/detail?id=118639 (not actually a bug just the way things are now)

For touch devices the only way to reliably use a browser terminal is to create a custom html/svg keyboard in the page itself (PWA/fullscreen) and then send keys yourself.

This is where "sendKey" comes in handy for me. Since I don't think a mac meta key is really needed for that, it is left out. (only mac can selectAll?/does a normal terminal do this at all (ctrl+a)? is selectAll only there to support macmeta+a..?)

closing this on my end for now - thank you for your time

[Tyriar]

@A-J-Bauer thanks for the PR, I created an issue out of it as it seems valuable #3581. Someone else might pick it up

[HALOCORE]

I am also looking for an API like SendKey to deal with virtual keyboard with specialized shortcuts (for vim/nginx/lxc, etc) on mobile phone for handy server maintainance. With SendKey it would be convenient to connect customized keyboards to this. Or are there existing APIs for this case? (sending key events to xterm.js or execute a command)

[andy380743909]

Any progress on this issue?