diff --git a/.gitignore b/.gitignore index ef29d45439f..b4af6c0b81e 100644 --- a/.gitignore +++ b/.gitignore @@ -4,6 +4,8 @@ .zebra-state/ # Nix configs shell.nix +# Docker compose env files +*.env # ---- Below here this is an autogenerated .gitignore using Toptal ---- # Created by https://www.toptal.com/developers/gitignore/api/firebase,emacs,visualstudiocode,rust,windows,macos diff --git a/book/src/user/docker.md b/book/src/user/docker.md index 82764e00c73..bf0f33d9704 100644 --- a/book/src/user/docker.md +++ b/book/src/user/docker.md @@ -2,7 +2,10 @@ The easiest way to run Zebra is using [Docker](https://docs.docker.com/get-docker/). -We've embraced Docker in Zebra for most of the solution lifecycle, from development environments to CI (in our pipelines), and deployment to end users. We recommend using `docker-compose` over plain `docker` CLI, especially for more advanced use-cases like running CI locally, as it provides a more convenient and powerful way to manage multi-container Docker applications. +We've embraced Docker in Zebra for most of the solution lifecycle, from development environments to CI (in our pipelines), and deployment to end users. + +> [!TIP] +> We recommend using `docker compose` sub-command over the plain `docker` CLI, especially for more advanced use-cases like running CI locally, as it provides a more convenient and powerful way to manage multi-container Docker applications. See [CI/CD Local Testing](#cicd-local-testing) for more information, and other compose files available in the [docker](https://github.com/ZcashFoundation/zebra/tree/main/docker) folder. ## Quick usage @@ -10,8 +13,25 @@ You can deploy Zebra for daily use with the images available in [Docker Hub](htt ### Ready to use image +Using `docker compose`: + +```shell +docker compose -f docker/docker-compose.yml up +``` + +With plain `docker` CLI: + ```shell -docker run --detach zfnd/zebra:latest +docker volume create zebrad-cache + +docker run -d --platform linux/amd64 \ + --restart unless-stopped \ + --env-file .env \ + --mount type=volume,source=zebrad-cache,target=/var/cache/zebrad-cache \ + -p 8233:8233 \ + --memory 16G \ + --cpus 4 \ + zfnd/zebra ``` ### Build it locally @@ -32,7 +52,7 @@ You're able to specify various parameters when building or launching the Docker For example, if we'd like to enable metrics on the image, we'd build it using the following `build-arg`: -> [!WARNING] +> [!IMPORTANT] > To fully use and display the metrics, you'll need to run a Prometheus and Grafana server, and configure it to scrape and visualize the metrics endpoint. This is explained in more detailed in the [Metrics](https://zebra.zfnd.org/user/metrics.html#zebra-metrics) section of the User Guide. ```shell @@ -63,6 +83,14 @@ cache_dir = "/var/cache/zebrad-cache" endpoint_addr = "127.0.0.1:9999" ``` +### Running Zebra with Lightwalletd + +To run Zebra with Lightwalletd, we recommend using the provided `docker compose` files for Zebra and Lightwalletd, which will start both services and connect them together, while exposing ports, mounting volumes, and setting environment variables. + +```shell +docker compose -f docker/docker-compose.yml -f docker/docker-compose.lwd.yml up +``` + ### CI/CD Local Testing To run CI tests locally, which mimics the testing done in our CI pipelines on GitHub Actions, use the `docker-compose.test.yml` file. This setup allows for a consistent testing environment both locally and in CI. diff --git a/book/src/user/lightwalletd.md b/book/src/user/lightwalletd.md index 230bb879e34..4688c89e519 100644 --- a/book/src/user/lightwalletd.md +++ b/book/src/user/lightwalletd.md @@ -6,17 +6,21 @@ recommend using use it in testing. Other `lightwalletd` forks have limited support, see the [Sync lightwalletd](#sync-lightwalletd) section for more info. +> [!NOTE] +> You can also use `docker` to run lightwalletd with zebra. Please see our [docker documentation](./docker.md#running-zebra-with-lightwalletd) for more information. + Contents: -- [Configure zebra for lightwalletd](#configure-zebra-for-lightwalletd) - - [JSON-RPC](#json-rpc) -- [Sync Zebra](#sync-zebra) -- [Download and build lightwalletd](#download-and-build-lightwalletd) -- [Sync lightwalletd](#sync-lightwalletd) -- [Run tests](#run-tests) -- [Connect wallet to lightwalletd](#connect-wallet-to-lightwalletd) - - [Download and build the cli-wallet](#download-and-build-the-cli-wallet) - - [Run the wallet](#run-the-wallet) +- [Running lightwalletd with zebra](#running-lightwalletd-with-zebra) + - [Configure zebra for lightwalletd](#configure-zebra-for-lightwalletd) + - [JSON-RPC](#json-rpc) + - [Sync Zebra](#sync-zebra) + - [Download and build lightwalletd](#download-and-build-lightwalletd) + - [Sync lightwalletd](#sync-lightwalletd) + - [Run tests](#run-tests) + - [Connect a wallet to lightwalletd](#connect-a-wallet-to-lightwalletd) + - [Download and build the cli-wallet](#download-and-build-the-cli-wallet) + - [Run the wallet](#run-the-wallet) ## Configure zebra for lightwalletd diff --git a/docker/.env b/docker/.env new file mode 100644 index 00000000000..2d96240f23e --- /dev/null +++ b/docker/.env @@ -0,0 +1,33 @@ +RUST_LOG=info +# This variable forces the use of color in the logs +ZEBRA_FORCE_USE_COLOR=1 +LOG_COLOR=true + +### +# Configuration Variables +# These variables are used to configure the zebra node +# Check the entrypoint.sh script for more details +### + +# The config file full path used in the Dockerfile. +ZEBRA_CONF_PATH=/etc/zebrad/zebrad.toml +# [network] +NETWORK=Mainnet +ZEBRA_LISTEN_ADDR=0.0.0.0 +# [consensus] +ZEBRA_CHECKPOINT_SYNC=true +# [state] +# Set this to change the default cached state directory +ZEBRA_CACHED_STATE_DIR=/var/cache/zebrad-cache +LIGHTWALLETD_DATA_DIR=/var/cache/lwd-cache +# [metrics] +METRICS_ENDPOINT_ADDR=0.0.0.0 +METRICS_ENDPOINT_PORT=9999 +# [tracing] +TRACING_ENDPOINT_ADDR=0.0.0.0 +TRACING_ENDPOINT_PORT=3000 +# [rpc] +RPC_LISTEN_ADDR=0.0.0.0 +# if ${RPC_PORT} is not set, it will use the default value for the current network +RPC_PORT=8232 + diff --git a/docker/Dockerfile b/docker/Dockerfile index 55b8f4da381..570642a7949 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -188,6 +188,7 @@ COPY --from=release /entrypoint.sh / RUN apt-get update && \ apt-get install -y --no-install-recommends \ ca-certificates \ + curl \ rocksdb-tools # Config settings for zebrad diff --git a/docker/docker-compose.lwd.yml b/docker/docker-compose.lwd.yml new file mode 100644 index 00000000000..7d8c56b1855 --- /dev/null +++ b/docker/docker-compose.lwd.yml @@ -0,0 +1,59 @@ +version: "3.8" + +services: + zebra: + ports: + - "8232:8232" # Opens an RPC endpoint (for lightwalletd and mining) + healthcheck: + start_period: 1m + interval: 15s + timeout: 10s + retries: 3 + test: ["CMD-SHELL", "curl --data-binary '{\"id\":\"curltest\", \"method\": \"getinfo\"}' -H 'content-type: application/json' 127.0.0.1:8232 || exit 1"] + + lightwalletd: + image: electriccoinco/lightwalletd + platform: linux/amd64 + depends_on: + zebra: + condition: service_started + restart: unless-stopped + deploy: + resources: + reservations: + cpus: "4" + memory: 16G + environment: + - LWD_GRPC_PORT=9067 + - LWD_HTTP_PORT=9068 + configs: + - source: lwd_config + target: /etc/lightwalletd/zcash.conf + uid: '2002' # Golang's container default user uid + gid: '2002' # Golang's container default group gid + mode: 0440 + volumes: + - litewalletd-data:/var/lib/lightwalletd/db + #! This setup with --no-tls-very-insecure is only for testing purposes + #! For production environments follow the guidelines here: https://github.com/zcash/lightwalletd#production-usage + command: > + --no-tls-very-insecure + --grpc-bind-addr=0.0.0.0:9067 + --http-bind-addr=0.0.0.0:9068 + --zcash-conf-path=/etc/lightwalletd/zcash.conf + --data-dir=/var/lib/lightwalletd/db + --log-file=/dev/stdout + --log-level=7 + ports: + - "9067:9067" # gRPC + - "9068:9068" # HTTP + +configs: + lwd_config: + # Change the following line to point to a zcash.conf on your host machine + # to allow for easy configuration changes without rebuilding the image + file: ./zcash-lightwalletd/zcash.conf + +volumes: + litewalletd-data: + driver: local diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml new file mode 100644 index 00000000000..fd1b2ccc5b1 --- /dev/null +++ b/docker/docker-compose.yml @@ -0,0 +1,45 @@ +version: "3.8" + +services: + zebra: + image: zfnd/zebra + platform: linux/amd64 + build: + context: ../ + dockerfile: docker/Dockerfile + target: runtime + restart: unless-stopped + deploy: + resources: + reservations: + cpus: "4" + memory: 16G + env_file: + - .env + #! Uncomment the `configs` mapping below to use the `zebrad.toml` config file from the host machine + #! NOTE: This will override the zebrad.toml in the image and make some variables irrelevant + # configs: + # - source: zebra_config + # target: /etc/zebrad/zebrad.toml + # uid: '2001' # Rust's container default user uid + # gid: '2001' # Rust's container default group gid + # mode: 0440 + volumes: + - zebrad-cache:/var/cache/zebrad-cache + ports: + # Zebra uses the following default inbound and outbound TCP ports + - "8233:8233" # Mainnet Network (for peer connections) + # - "8232:8232" # Opens an RPC endpoint (for wallet storing and mining) + # - "18233:18233" # Testnet Network + # - "9999:9999" # Metrics + # - "3000:3000" # Tracing + +configs: + zebra_config: + # Change the following line to point to a zebrad.toml on your host machine + # to allow for easy configuration changes without rebuilding the image + file: ../zebrad/tests/common/configs/v1.0.0-rc.2.toml/ + +volumes: + zebrad-cache: + driver: local diff --git a/docker/test.env b/docker/test.env index 54dcdc1ae95..fd2a7c876b7 100644 --- a/docker/test.env +++ b/docker/test.env @@ -1,36 +1,26 @@ -RUST_LOG=info -# This variable forces the use of color in the logs -ZEBRA_FORCE_USE_COLOR=1 -LOG_COLOR=true - ### # Configuration Variables # These variables are used to configure the zebra node # Check the entrypoint.sh script for more details ### -# Path and name of the config file. These two have defaults set in the Dockerfile. -ZEBRA_CONF_PATH=/etc/zebrad/zebrad.toml +# Set this to change the default log level (must be set at build time) +RUST_LOG=info +# This variable forces the use of color in the logs +ZEBRA_FORCE_USE_COLOR=1 +LOG_COLOR=true +# Path to the config file. This variable has a default set in entrypoint.sh +# ZEBRA_CONF_PATH=/etc/zebrad/zebrad.toml # [network] NETWORK=Mainnet -ZEBRA_LISTEN_ADDR=0.0.0.0 -# [consensus] -ZEBRA_CHECKPOINT_SYNC=true # [state] # Set this to change the default cached state directory ZEBRA_CACHED_STATE_DIR=/var/cache/zebrad-cache LIGHTWALLETD_DATA_DIR=/var/cache/lwd-cache -# [metrics] -METRICS_ENDPOINT_ADDR=0.0.0.0 -METRICS_ENDPOINT_PORT=9999 # [tracing] LOG_COLOR=false TRACING_ENDPOINT_ADDR=0.0.0.0 TRACING_ENDPOINT_PORT=3000 -# [rpc] -RPC_LISTEN_ADDR=0.0.0.0 -# if ${RPC_PORT} is not set, it will use the default value for the current network -# RPC_PORT= #### # Test Variables diff --git a/docker/zcash-lightwalletd/zcash.conf b/docker/zcash-lightwalletd/zcash.conf new file mode 100644 index 00000000000..dc2a0238fc5 --- /dev/null +++ b/docker/zcash-lightwalletd/zcash.conf @@ -0,0 +1,5 @@ +rpcbind=zebra +rpcport=8232 +rpcuser=zcashrpc +rpcpassword=changeme +testnet=0