subsys: shell rework

[jakub-uC]

Me and @nordic-krch have reworked Zephyr shell Task 8872.
New one has a lot of useful features that were missing in the old one.

We have changed commands concept.

1. Shell commands are now organized in a tree structure and grouped into the following types:

• Root command (level 0): Kept in the section variable area. Root command is always static.
• Static subcommand (level > 0): Number and syntax must be known during compile time. Kept in the software module.
• Dynamic subcommand (level > 0): Number and syntax does not need to be known during compile time. Created in the software module.

On one command level all commands must be the same type: static or dynamic.
An example usage of dynamic commands concept is described here:
https://devzone.nordicsemi.com/b/blog/posts/new-command-line-interface-part-1

2. Commands execution
   Each command or subcommand on each level can either have or not have a handler. Shell executes the handler that is found deepest in the command tree and further subcommands (without a handler) are passed as arguments. Characters within parentheses are treated as one argument. Each command or subcommand without a handler can be treated as an argument, or it can still have a subcommand with a handler and there is no problem for the user to execute it.
   It is recommended to use subcommands over options. Options apply mainly in the case when an argument with '-' or "--" is requested.

3. Commands help
   Every user-defined command, subcommand, or option can have its own help description. The help for commands and subcommands can be created with respective macros: SHELL_CMD_REGISTER, SHELL_CMD. In addition, you can define options for commands or subcommands using the macro SHELL_OPT. By default, each and every command or subcommand has these two options implemented: "-h" and "--help".
   In order to add help functionality to a command or subcommand, you must call the help function shell_help_print inside of a command handler.

New features:

• Multi-instance - you can have independent shells on different transmission mediums

• Multiline commands - Shell will correctly print and allow to edit commands longer than 1 line.

• Integration with Log module - you can read logs on the shell screen, filter them dynamically (activate logs only for needed modules), suspend or resume. At the same time you can type, edit or execute a command.

• Smart completion with the Tab key. One can prompt and partially/fully complete commands or its subcommands.

• Commands history - Executed commands can be listed by history command or recalled with up / down arrows.

• Built-in commands - Shell related commands already implemented and available for the user:

1. clear - clears the screen
2. history - prints recently called commands
3. resize - command shall be called each time terminal width has changed. It will ensure correct text formatting (currently it is only working with UART flow control on)
4. shell colors - switch on / off colored syntax
5. shell echo - switch on / off shell echo
6. shell backspace_mode - allows to adjust backspace escape code during runtime.
7. shell stats - gives an information about lost (not printed) logs

• Easy command edition with buttons: Tab/Backspace/Delete/Arrows/Home/End

• Meta Keys:

1. ctrl + a - Moves the cursor to the beginning of the line.
2. ctrl + c - Preserves the last command on the screen and starts a new command in a new line.
3. ctrl + e - Moves the cursor to the end of the line.
4. ctrl + l - Clears the screen and leaves the currently typed command at the top of the screen.
5. ctrl + u - Clears the currently typed command.
6. ctrl + w - Removes the word or part of the word to the left of the cursor. Words separated by period instead of space are treated as one word.

• Wildcards support for: * and ?

• Kconfig configuration to optimize resources usage.

[jakub-uC]

@carlescufi FYI

[nordic-krch]

@jukkar @jhedberg please take a look. Shell is ready thus bluetooth and network commands adaptation can start. Checkout samples/subsys/shell to see how to port.

[codecov-io]

Codecov Report

Merging #9362 into master will decrease coverage by 0.76%.
The diff coverage is 38.69%.

@@            Coverage Diff             @@
##           master    #9362      +/-   ##
==========================================
- Coverage   52.94%   52.18%   -0.77%     
==========================================
  Files         214      212       -2     
  Lines       26163    25924     -239     
  Branches     5639     5570      -69     
==========================================
- Hits        13853    13528     -325     
- Misses      10037    10133      +96     
+ Partials     2273     2263      -10

Impacted Files	Coverage Δ
subsys/shell/shell_service.c	100% <ø> (ø)	⬆️
include/misc/dlist.h	92.59% <ø> (ø)	⬆️
subsys/logging/log_core.c	67.66% <0%> (-2.57%)	⬇️
subsys/logging/log_output.c	2.64% <0%> (-62.96%)	⬇️
subsys/shell/legacy_shell.c	47.16% <47.16%> (ø)
subsys/random/rand32_timer.c	0% <0%> (-100%)	⬇️
lib/cmsis_rtos_v1/cmsis_kernel.c	0% <0%> (-80%)	⬇️
lib/cmsis_rtos_v1/cmsis_msgq.c	35.89% <0%> (-39.72%)	⬇️
lib/cmsis_rtos_v1/cmsis_mailq.c	50% <0%> (-30.77%)	⬇️
subsys/net/ip/ipv6.c	37.09% <0%> (-21.85%)	⬇️
... and 91 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 732b45c...9b8682c. Read the comment docs.

[Qbicz]

Great work. I like the dynamic command sample. Some minor comments from me.

[Qbicz]

Excess empty line, please also check between functions.

[Qbicz]

Can you name this 64?

[Qbicz]

Unsigned or size_t

[jukkar]

LGTM

[jukkar]

Just tried this with make menuconfig and the shell support looks really bad as all the config options are in the "root". The Kconfig seems to be missing menu "Shell support" or similar command.

[jakub-uC]

Good point, I will update it.

[jakub-uC]

@jukkar : done