New shell subsystem

[nordic-krch]

New shell subsystem will be ported from nRF5 SDK (known there as nrf_cli). It has following features:

• multi-instance
• user friendly features: autocompletion, wildcards, coloring, metakeys (arrows, backspace, ctrl+u, etc.), history
• integration with logger subsystem - logs are multiplexed with data outputed by the shell ensuring that they do not interfere
• shell transport generic api - indenpendent of the shell
• static command tree (root command links to subcommands, subcommands can link to another subcommands set)
• dynamic subcommands - autocompletion can use dynamic data not known during compilation

New shell subsystem shall be merged when legacy shell is totally replaced targeting 1.14 release.

Description	Task	Assignee	PR
Bluetooth	#8873 #9871		#10239
FS	#8874	@nordic-krch	#10103
net/ip	#8875	@jukkar	PR:#9825
net/l2/ieee802154	#8876	@jukkar	PR:#9825
net/l2/bluetooth	#8877	@jukkar	PR:#9825
net/l2/wifi	#8878	@jukkar	PR:#9825
net/lib/openthread	#8879	@jukkar	PR:#9825
shell/modules/kernel_service	#8880	@nordic-krch	PR: #9363
samples/subsys/shell	#8881	@jarz-nordic	PR: #9362
samples/net	#8882	@jukkar	PR:#9825
samples/drivers/flash_shell	#8883	@jarz-nordic	PR: #10019
samples/mpu/mpu_test	#8884	@jarz-nordic	PR: #10608

[nordic-krch]

fyi: @carlescufi @jarz-nordic @pizi-nordic @nashif

[pfalcon]

I'm all for detailed bug tickets, but her were have ~20 absolutely identical tickets of "Adapt xxx to new shell subsystem". What's the purpose of them? Why just having checkboxes in this master ticket isn't enough?

[nordic-krch]

@pfalcon I agree that this may not look nice but introducing this new console will require adaption work which will be split across multiple contributors. Separate tasks can help in terms of taking responsibility.

[koffes]

Looking forward to these new features. Will multiple transports be supported so that SYS_LOG may be routed to RTT ch. 0 and the shell be accessible over UART or RTT ch. 1 at the same time?

[jakub-uC]

@koffes yes it will be possible. But why would you like to have separate channel for SYS_LOG and shell? New shell will allow you to have SYS_LOG and shell on one RTT channel.

[koffes]

@jarz-nordic During debugging, we often have significant SYS_LOG output, whilst we want to be able to use the shell. Reading from shell/writing to shell in the middle of all this log "noise" is difficult.

[jakub-uC]

@koffes : Ok, I get it.
Btw. in the new shell you will have possibility to suspend logging, or change filtering criteria during runtime via shell.

I just would like to a bit update my previous answer. In the initial commit we will provide only UART backend. To use shell via RTT, one will need to help us and add RTT backend. It shall not be big deal. Most of work will be copy-paste from UART backend.

[jakub-uC]

@koffes : New shell is in upstream master.

[jakub-uC]

All examples have been migrated.