node-snapshots

Stores the snapshots of Ethereum Execution and Consensus clients

Overview

This tool allows you to manage Ethereum/Gnosis nodes running in docker containers snapshots by creating backups of specified directories, uploading those backups to Google Cloud Storage (GCS), and facilitating the recovery process. The tool supports running multiple node backups in parallel for efficiency.

Features

• Node Snapshot: Create tarballs of specified directories with container stop.
• Upload to Google Cloud Storage: Upload the created tarballs to a specified GCS bucket.
• Parallel Execution: Take snapshots of multiple containers in parallel.
• Recovery: Download and extract snapshots from GCS to recover node data.

Prerequisites

• Docker: Ensure Docker is installed and running on your machine.
• Google Cloud Storage: You need a Google Cloud project with a GCS bucket and service account credentials.
• Python 3.12+: The script uses Python 3 and requires the installation of the necessary libraries.

Configuration

Configuration File (config.toml)

This tool uses a .toml configuration file to specify the Docker containers to back up, as well as the Google Cloud Storage details. Here’s an example configuration:

# Docker host configuration
docker_host = "unix:///var/run/docker.sock"

# Google Cloud service account credentials
google_credentials_json = "google-credentials.json"

# Google Cloud Storage bucket name
bucket_name = "node-snapshots-bucket"

# Docker containers and their paths for snapshot management
[[docker_containers]]
container_name = "nethermind"
data_path = "/data/nethermind"
tar_name = "nethermind.tar"
bucket_path = "mainnet/nethermind.tar"
recovery_path = "/data/recovered/nethermind"

[[docker_containers]]
container_name = "lighthouse"
data_path = "/data/lighthouse"
tar_name = "nethermind.tar"
bucket_path = "mainnet/lighthouse.tar"
recovery_path = "/data/recovered/lighthouse"

Configuration Fields

• docker_host: The location of your Docker daemon (e.g., "unix:///var/run/docker.sock" for Linux or macOS).
• google_credentials_json: Path to the Google Cloud service account credentials JSON file.
• bucket_name: The name of your GCS bucket to upload snapshots to.
• docker_containers:
  • container_name: The name of the Docker container to back up.
  • data_path: Path inside the container to back up (relative or absolute).
  • tar_name: Name of the tarball file to create.
  • bucket_path: Path in the GCS bucket where the tarball will be uploaded.
  • recovery_path: Path on the local filesystem where the tarball will be extracted during recovery.

Usage

1. Take Snapshots of Docker Containers

Run the tool to take snapshots of the Docker containers specified in the configuration file. This will create tarballs of the data_path directories, upload them to GCS, and clean up after the operation.

python snapshot.py --config path_to_your_config.toml

2. Recovery of Docker Containers

To recover the data from the snapshot, the tool can download the tarball from GCS and extract it to the specified recovery_path.

The script will:

1. Check if the recovery directory exists.
2. If the directory is not empty, ask for confirmation before deleting the existing files.
3. Download the tarball from GCS.
4. Extract the tarball to the recovery_path.

To trigger the recovery process, simply run the same tool, and it will automatically download and extract the snapshots.

python recovery.py --config path_to_your_config.toml

Example Workflow

1. Configuration: Set up the config.toml file with the paths to Docker containers and the corresponding Google Cloud Storage bucket details.
2. Run Backup: Run the backup script to create tarballs of the Docker container data and upload them to Google Cloud Storage.
3. Recovery: Run the recovery script to download and extract the snapshot tarballs into the desired recovery directories.

Example Docker run

docker run --rm -ti \
  -v /root/node-snapshots:/config \
  -v /data/mainnet:/data \
  -v /var/run/docker.sock:/var/run/docker.sock \
  europe-west4-docker.pkg.dev/stakewiselabs/public/node-snapshots:v0.0.2 \
  snapshot.py -c /config/config.toml

Troubleshooting

• Google Cloud Authentication: Make sure the google-credentials.json file points to a valid Google Cloud service account with the necessary permissions to access your GCS bucket.
• Docker Issues: Ensure that the Docker daemon is running and that the container names in the configuration file match the actual running containers.
• Permission Issues: Verify that the user running the script has permission to write to the specified paths (e.g., the local recovery path and the GCS bucket).