Improvements in argument parsing

[phormio]

I propose these changes:

1. Stop printing the banner (e.g. Gossip 0.14.0-unstable-647ed422) for Gossip subcommands, except when the subcommand is help.
2. Make log messages go to standard error instead of standard output.
3. In Gossip's user documentation, explain the effect of putting RUST_LOG=off and RUST_LOG=error in the environment.
4. Eliminate the rapid subcommand and replace it with --rapid, so one would run gossip --rapid or gossip --rapid rebuild_fof. Optionally, retain rapid for a year (say), for backwards compatibility.
5. Remove the oneshot subcommand.
6. Add a --descriptions option for the help subcommand, to print the descriptions of all the subcommands.

In case (6) isn't clear: the effect of gossip help --descriptions would be similar to:

RUST_LOG=off gossip help |
awk '/^ / { print $1 }' |
while read subcommand; do
  # Pipe into sed to discard Gossip's banner.
  RUST_LOG=off gossip help "$subcommand" | sed 1d
done

So the output would be something like:

gossip oneshot {depends}
    temporary oneshot action
gossip add_person_list <listname>
    add a new person list with the given name
gossip backdate_eose 
    backdate last_general_eose_at by 24 hours for every relay. This will usually cause gossip to refetch recent things.
gossip bech32_decode <bech32string>
    decode the bech32 string.
gossip bech32_encode_naddr <kind> <pubkey> <d> [<relayurl>, ...]
    encode an event address (parameterized replaceable event link).
gossip clear_timeouts 
    clear relay avoidance timeouts.
gossip decrypt <pubkey> <ciphertext>
    decrypt the ciphertext from the pubkeyhex.
gossip delete_by_kind <kind>
    deletes all events of the given kind (be careful!)
gossip delete_spam_by_content <kind> <unixtime_since> <substring>
    delete all feed-displayable events with content matching the substring (including giftwraps)
gossip delete_relay <relayurl>
    delete a relay record from storage. Be aware any event referencing it will cause it to be recreated.
gossip dpi <dpi>
    override the DPI setting
gossip disable_relay <relayurl>
    Set a relay rank to 0 so it will never connect, and also hide form thie list. This is better than delete because deleted relays quickly come back with default settings.
gossip dump_handlers 
    print all the web-based event handlers
gossip events_of_kind <kind>
    print IDs of all events of kind=<kind>
gossip events_of_pubkey <pubkey>
    print IDs of all events from <pubkeyhex>
gossip events_of_pubkey_and_kind <pubkey> <kind>
    print IDs of all events from <pubkeyhex> of kind=<kind>
gossip export_encrypted_key 
    Export the encrypted private key
gossip force_migration_level <level>
    DANGEROUS! force storage migration level
gossip giftwraps 
    Decrypt all of your giftwraps
gossip help <command>
    show this list
gossip import_encrypted_private_key <ncryptsec>
    import encrypted private key
gossip import_event <event_json>
    import and process a JSON event
gossip login 
    login on the command line before starting the gossip GUI
gossip offline 
    start gossip in offline mode
gossip print_event <idhex>
    print the event (in JSON) from the database that has the given id
gossip print_followed 
    print every pubkey that is followed
gossip print_muted 
    print every pubkey that is muted
gossip print_person_lists 
    print every pubkey in every person list
gossip print_person <pubkey>
    print the given person
gossip print_person_relays <pubkey>
    print all the person-relay records for the given person
gossip print_relay <url>
    print the relay record
gossip print_relays 
    print all the relay records
gossip print_seen_on <idhex>
    print the relays the event was seen on
gossip rapid 
    Use much faster disk access. A crash can corrupt your local data, unless your filesystem preserves write ordering
gossip reaction_stats 
    Show statistics on reactions
gossip rebuild_fof 
    Rebuild friends-of-friends (will rebuild next time gossip starts)
gossip rebuild_indices 
    Rebuild all event-related indices
gossip rename_person_list <number> <newname>
    Rename a person list
gossip reprocess_recent 
    Reprocess events that came during the last 24 hours
gossip reprocess_relay_lists 
    Reprocess relay lists (including kind 3 contents)
gossip theme <dark | light>
    Start gossip with the selected theme
gossip ungiftwrap <idhex>
    Unwrap the giftwrap event with the given ID and print the rumor (in JSON)
gossip verify <idhex>
    Verify if the given event signature is valid
gossip verify_json <event_json>
    Verify if the passed in event JSON's signature is valid
gossip wgpu_renderer true | false
    Enable/Disable the WGPU rendering backend

And here are the respective reasons for these changes:

1. The banner makes it impossible to run commands such as gossip print_relays | jq -r .url
2. Essentially the same as the previous reason.
3. My guess is that many Gossip users don't know this. (I didn't know it until recently.)
4. The current way of doing things is non-standard.
5. I spent time investigating oneshot only to discover it doesn't do anything.
6. It's a useful convenience for something I have found myself wanting to do.

[mikedilger]

Thanks for the report. I"ll look into it when I get back tomorrow.

[mikedilger]

This is good stuff. I'm making many of these changes.

Do you use 'rapid'? Does it make a difference in your case?

oneshot is for developer use only. I've changed the description and removed it from help.

[mikedilger]

How about help just always showing the descriptions? I don't see why not. Help output is unlikely to be programmatically parsed.

[phormio]

Do you use 'rapid'?

No, I've never used rapid.

As far as I can remember, what led me to think about rapid was that I was looking at the code, wondering how I would make some of the changes described above, and I thought they would be easier if rapid were changed to --rapid.

[phormio]

How about help just always showing the descriptions?

The only reason not to do this would be if you foresee the output getting very long.

Help output is unlikely to be programmatically parsed.

If you're thinking that nobody out there has code that would be broken by a change in the format of gossip help: I agree.

[mikedilger]

Rapid was actually already parsed separately, it just didn't look like a command line option. So you can in older versions say "gossip rapid help" and it will print help. I just changed it to also accept "gossip --rapid".

I think I've done all of this on unstable, so please check and close if you are happy.

[phormio]

I can't check it now but should do it within 24 hours.

[phormio]

I looked at the relevant commits on the unstable branch. I also built Gossip from the HEAD of unstable (10746c9) and ran target/release/gossip help. A few remarks:

1. There is no reason to indent the output of gossip help now that the banner is gone.
2. --rapid is not mentioned in the output of gossip help.
3. I don't know if you disagree with using RUST_LOG=off, but it's not mentioned in the new documentation.
4. It might be worth linking to the full documentation for RUST_LOG: https://docs.rs/env_logger/latest/env_logger/#enabling-logging.
5. I was going to put this in a GitHub issue, but you might prefer to have it now: the documentation for the help subcommand itself could do with a change. Details below.

The current documentation of the help subcommand is:

  help <command>
      show this list

It would be better to have something like:

  help [<command>]
      Show documentation of <command> if <command> is specified,
      otherwise documentation of all commands.

I included a line break for readability, but for consistency, you should probably put "Show [...] all commands" on a single line.

[phormio]

Perhaps this is off-topic, but I'll say it anyway. If I run target/release/gossip --rapid print_muted, I get a message at the warn level saying "Storage using rapid config [...]". This is not new behavior, and I'm not saying it's wrong, but it could be argued that it's not ideal to warn the user about something he asked for.

[mikedilger]

I didn't know about RUST_LOG=off and hadn't confirmed it is a thing. Now I have. I've made some of these changes, but a bit differently in some cases.

[phormio]

I looked at the 3 new commits on the unstable branch.

I built Gossip at ce6a072 and ran it a few times.

The changes are more or less in line with my second batch of suggestions.

I don't have any more suggestions.

Thanks for making it easier to pipe Gossip's output into jq.