Open the database in a blocking tokio thread

[teor2345]

Motivation

Zebra can take a long time to re-open the database after the initial sync:

2022-07-25T03:50:05.294821Z WARN {net="Main"}: zebra_chain::diagnostic: opening finalized state database code took a long time to execute execution_time="1m 8s" module_path=Some("zebra_state::service") line=Some(172)

We should do this work in a blocking thread, so it doesn't block the rest of the tokio executor.

Some RocksDB documentation warns that the first read can also be slow, so we might want to do it in the same blocking call.

Designs

• Open the database and do the first database read using tokio::spawn_blocking() or tokio::block_in_place()
• Log an info-level message before opening the database, so that users know it might take a while

Related Work

• Speed up opening the database #4822

[conradoplg]

I wonder if some of these suggestions can also reduce the opening time

[oxarbitrage]

@conradoplg , there is another ticket to try to apply some of those: #4822

[teor2345]

This is something we want to do eventually, but it's not urgent.

[teor2345]

We might want to do this usability fix some time during the release candidate series, but it's not a blocker.

[teor2345]

@arya2 it's ok to do the spawn_blocking() in zebrad::commands::start or zebra_state::init(), whatever is easier.