From ee56a82a01e0c7b2a57df8013ddc845b324b21de Mon Sep 17 00:00:00 2001 From: teor Date: Mon, 14 Feb 2022 10:39:48 +1000 Subject: [PATCH 1/6] doc(README): remove completed Zebra goals --- README.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index b8ca45d7da0..fcb82fd101e 100644 --- a/README.md +++ b/README.md @@ -82,8 +82,6 @@ The goals of the beta release series are for Zebra to act as a fully validating for all active consensus rules as of NU5 activation. Currently, Zebra validates all of the documented Zcash consensus rules, but it may not validate any: - -#### Other - Undocumented rules derived from Bitcoin - Undocumented network protocol requirements @@ -191,19 +189,18 @@ If you forcibly terminate it, or it panics, any incomplete changes will be rolle ## Future Work -In 2021, we intend to finish NU5 validation, start adding RPC support and start adding wallet integrations. +In 2022, we intend to start adding RPC support and start adding wallet integrations. This phased approach allows us to test Zebra's independent implementation of the consensus rules, before asking users to entrust it with their funds. Features: -- Full consensus rule validation -- Wallet functionality - RPC functionality +- Wallet functionality Performance and Reliability: - Reliable syncing on Testnet - Reliable syncing under poor network conditions -- Batch verification +- Additional batch verification - Performance tuning Currently, the following features are out of scope: From 8c2639590ef8ccf30875f1febcc535a23a1e8cac Mon Sep 17 00:00:00 2001 From: teor Date: Mon, 14 Feb 2022 10:44:23 +1000 Subject: [PATCH 2/6] doc(README): docker now uses bullseye --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index fcb82fd101e..53e0987d625 100644 --- a/README.md +++ b/README.md @@ -124,7 +124,7 @@ The *latest* [GitHub Runners](https://docs.github.com/en/actions/using-github-ho - Ubuntu Docker: -- Debian Buster +- Debian Bullseye Zebra's tests can take over an hour, depending on your machine. We're working on making them faster. From 9998af01df662ad9ec498c457569272066082f75 Mon Sep 17 00:00:00 2001 From: teor Date: Mon, 14 Feb 2022 10:40:31 +1000 Subject: [PATCH 3/6] doc(README): clarify and expand disk requirements --- README.md | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 53e0987d625..5d75d2d9e5b 100644 --- a/README.md +++ b/README.md @@ -113,7 +113,7 @@ For more detailed instructions, refer to the [documentation](https://zebra.zfnd. The recommended requirements for compiling and running `zebrad` are: - 4+ CPU cores - 16+ GB RAM -- 50GB+ available disk space for finalized state +- 50GB+ available disk space for building binaries and storing finalized state - 100+ Mbps network connections We continuously test that our builds and tests pass on: @@ -175,6 +175,23 @@ You can set `ZEBRA_SKIP_NETWORK_TESTS=1` to skip the network tests. Zebra may be unreliable on Testnet, and under less-than-perfect network conditions. See our [roadmap](#future-work) for details. +### Disk Usage + +Zebra uses up to 40 GB of space for cached mainnet data, +and 10 GB of space for cached testnet data. + +RocksDB cleans up outdated data periodically, +and when the database is closed and re-opened. + +#### Disk Troubleshooting + +Zebra's state commits changes using RocksDB database transactions. + +If you forcibly terminate Zebra, or it panics, +any incomplete changes will be rolled back the next time it starts. + +So Zebra's state should always be valid, unless your OS or disk hardware is corrupting data. + ## Known Issues There are a few bugs in Zebra that we're still working on fixing: @@ -184,9 +201,6 @@ There are a few bugs in Zebra that we're still working on fixing: - [Interrupt handler does not work when a blocking task is running #1351](https://github.com/ZcashFoundation/zebra/issues/1351) - Zebra should eventually exit once the task finishes. Or you can forcibly terminate the process. -Zebra's state commits changes using database transactions. -If you forcibly terminate it, or it panics, any incomplete changes will be rolled back the next time it starts. - ## Future Work In 2022, we intend to start adding RPC support and start adding wallet integrations. From 3c876ea1f98ec634b44b2432f5e6f2a81ead99b9 Mon Sep 17 00:00:00 2001 From: teor Date: Mon, 14 Feb 2022 10:43:32 +1000 Subject: [PATCH 4/6] doc(README): add network latency requirement Also note extra network usage after database format changes. --- README.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5d75d2d9e5b..b01f37996e8 100644 --- a/README.md +++ b/README.md @@ -161,9 +161,15 @@ By default, Zebra uses the following inbound TCP listener ports: - 8233 on Mainnet - 18233 on Testnet -`zebrad`'s typical network usage is: +Zebra needs some peers which have a round-trip latency of 2 seconds or less. +If this is a problem for you, please +[open a ticket.](https://github.com/ZcashFoundation/zebra/issues/new/choose) + +`zebrad`'s typical mainnet network usage is: - Initial sync: 30 GB download -- Ongoing updates: 10-50 MB upload and download per day, depending on peer requests +- Ongoing updates: 10-100 MB upload and download per day, depending on peer requests + +Zebra also performs an initial sync every time its internal database version changes. For more detailed information, refer to the [documentation](https://zebra.zfnd.org/user/run.html). From 820886185740bbc73fcc2caca52e1eb3943735ab Mon Sep 17 00:00:00 2001 From: teor Date: Mon, 14 Feb 2022 11:02:42 +1000 Subject: [PATCH 5/6] doc(run): de-duplicate README info --- book/src/user/run.md | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/book/src/user/run.md b/book/src/user/run.md index 520e78fad47..91959b50a04 100644 --- a/book/src/user/run.md +++ b/book/src/user/run.md @@ -20,9 +20,8 @@ structure, and documentation for all of the config options can be found ## Network Ports and Data Usage -By default, Zebra uses the following inbound TCP listener ports: -- 8233 on Mainnet -- 18233 on Testnet +`zebrad`'s default ports and network usage are +[documented in the README.](https://github.com/ZcashFoundation/zebra#network-ports-and-data-usage) If Zebra is configured with a specific [`listen_addr`](https://doc.zebra.zfnd.org/zebra_network/struct.Config.html#structfield.listen_addr), it will advertise this address to other nodes for inbound connections. @@ -31,10 +30,6 @@ Zebra makes outbound connections to peers on any port. But `zcashd` prefers peers on the default ports, so that it can't be used for DDoS attacks on other networks. -`zebrad`'s typical network usage is: -- initial sync: 30 GB download -- ongoing updates: 10-50 MB upload and download per day, depending on peer requests - The major constraint we've found on `zebrad` performance is the network weather, especially the ability to make good connections to other Zcash network peers. From 962b059140c4e25598cc3dae93dd429e30362c97 Mon Sep 17 00:00:00 2001 From: teor Date: Mon, 14 Feb 2022 11:03:04 +1000 Subject: [PATCH 6/6] doc(run): speed up Zebra's performance --- book/src/user/run.md | 60 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/book/src/user/run.md b/book/src/user/run.md index 91959b50a04..7b585477b87 100644 --- a/book/src/user/run.md +++ b/book/src/user/run.md @@ -33,3 +33,63 @@ so that it can't be used for DDoS attacks on other networks. The major constraint we've found on `zebrad` performance is the network weather, especially the ability to make good connections to other Zcash network peers. +Zebra needs some peers which have a round-trip latency of 2 seconds or less. +If this is a problem for you, please let us know! + +## Improving Performance + +Zebra usually syncs in 4-12 hours, depending on the network connection. + +If you're having trouble syncing, try the following config changes: + +### Release Build + +Make sure you're using a release build on your native architecture. + +If you're using an ARM machine, +[install the Rust compiler for ARM](https://rust-lang.github.io/rustup/installation/other.html). +If you build using the x86_64 tools, Zebra might run really slowly. + +Run a release build using the +[`cargo install` command from the README.](https://github.com/ZcashFoundation/zebra#build-and-run-instructions) + +### Syncer Lookahead Limit + +If your connection is slow, try +[downloading fewer blocks at a time](https://doc.zebra.zfnd.org/zebrad/config/struct.SyncSection.html#structfield.lookahead_limit): + +```toml +[sync] +lookahead_limit = 1000 +max_concurrent_block_requests = 25 +``` + +### Peer Set Size + +If your connection is slow, try [connecting to fewer peers](https://doc.zebra.zfnd.org/zebra_network/struct.Config.html#structfield.peerset_initial_target_size): + +```toml +[network] +peerset_initial_target_size = 25 +``` + +### Turn off debug logging + +Zebra logs at info level by default. + +If Zebra is slow, make sure it is logging at info level: + +```toml +[tracing] +filter = 'info' +``` + +Or restrict debug logging to a specific Zebra component: + +```toml +[tracing] +filter = 'info,zebra_network=debug' +``` + +If you keep on seeing multiple info logs per second, please +[open a bug.](https://github.com/ZcashFoundation/zebra/issues/new/choose)