Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forest tool migration of forest-cli subcommands #3336

Merged
merged 60 commits into from
Aug 18, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
b429561
Move DB subcommands to forest-tool
elmattic Aug 7, 2023
f1e07e4
Add hidden commands to help users migrating
elmattic Aug 8, 2023
71b4230
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 8, 2023
175d282
Fix lints
elmattic Aug 8, 2023
f97f070
Migrate config subcommand
elmattic Aug 8, 2023
2368ca7
Migrate fetch-params subcommand
elmattic Aug 8, 2023
2f46a5f
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 8, 2023
ae61c2f
Move Config to commands that require it
elmattic Aug 8, 2023
d13a849
Migrate snapshot fetch and validate subcommands
elmattic Aug 8, 2023
c6df6ae
Remove string manipulations
elmattic Aug 8, 2023
355aa7e
Migrate archive info subcommand
elmattic Aug 8, 2023
11c0442
Remove all duplicated strings
elmattic Aug 8, 2023
cf05e60
Use dedicated flag
elmattic Aug 8, 2023
f4dbb70
Rename DB clean subcommand
elmattic Aug 9, 2023
2405b4e
Add optional config file
elmattic Aug 9, 2023
55ad715
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 9, 2023
5bacfc8
Update harness script
elmattic Aug 9, 2023
a986ee4
Update other scripts
elmattic Aug 9, 2023
7d87792
Use toml file instead
elmattic Aug 9, 2023
ce05ce0
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 9, 2023
45c9d95
Add back db clean and db stats subcommands
elmattic Aug 9, 2023
2b513be
Update calls to snapshot fetch
elmattic Aug 9, 2023
fa26642
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 9, 2023
5910f10
Remove forest-tool db subcommands (will be in a dedicated PR)
elmattic Aug 9, 2023
035f5d4
Remove second pattern
elmattic Aug 9, 2023
7cfb7b0
Update CHANGELOG
elmattic Aug 9, 2023
6ff4c34
Fix shell script
elmattic Aug 9, 2023
fb3b711
Fix: chain flag is not needed for snapshot validate
elmattic Aug 9, 2023
9eed518
Fix fetch params tests
elmattic Aug 9, 2023
7a8f494
Revert config dump subcommand (will be in its own PR)
elmattic Aug 9, 2023
048312b
Update CHANGELOG
elmattic Aug 9, 2023
4da33d3
Update src/cli/subcommands/archive_cmd.rs
elmattic Aug 10, 2023
71108f7
Update src/cli/subcommands/db_cmd.rs
elmattic Aug 10, 2023
76b23f3
Update src/cli/subcommands/mod.rs
elmattic Aug 10, 2023
23de902
Update src/cli/subcommands/mod.rs
elmattic Aug 10, 2023
2ec85e2
Revert useless change
elmattic Aug 10, 2023
27eb1be
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 10, 2023
bd3e904
Update src/tool/subcommands/mod.rs
elmattic Aug 10, 2023
db6e18c
Update src/cli/subcommands/snapshot_cmd.rs
elmattic Aug 10, 2023
eb91628
Maintain order
elmattic Aug 10, 2023
42aee09
Fix last commit
elmattic Aug 10, 2023
ae2202a
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 10, 2023
1ac60a2
Change ordering
elmattic Aug 10, 2023
9e1d992
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 10, 2023
2fce1a1
Formatting
elmattic Aug 10, 2023
3b9fd28
Update README
elmattic Aug 10, 2023
26153f0
Fix formatting
elmattic Aug 10, 2023
16b7c53
Update the release git workflow
elmattic Aug 10, 2023
ef95096
Update Dockerfile
elmattic Aug 10, 2023
ef5f8b1
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 11, 2023
7ebc868
`forest-cli db` subcommands migration (#3355)
elmattic Aug 11, 2023
13774ab
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 11, 2023
2e9df3e
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 11, 2023
16765af
Merge branch 'elmattic/forest-tool-migration' of https://github.com/C…
elmattic Aug 11, 2023
602f0de
Replace macro by a function
elmattic Aug 11, 2023
1ecce48
Add check to alpine Dockerfile
elmattic Aug 11, 2023
86076c1
Add tracking issue
elmattic Aug 11, 2023
bcf4861
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 14, 2023
92a7e5f
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 17, 2023
f73094b
Merge branch 'main' into elmattic/forest-tool-migration
elmattic Aug 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion src/cli/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ where
Subcommand::Sync(cmd) => cmd.run(config).await,
Subcommand::Mpool(cmd) => cmd.run(config).await,
Subcommand::State(cmd) => cmd.run(config).await,
Subcommand::Config(cmd) => cmd.run(&config, &mut std::io::stdout()),
Subcommand::Send(cmd) => cmd.run(config).await,
Subcommand::Info(cmd) => cmd.run(config, opts).await,
Subcommand::DB(cmd) => cmd.run(&config).await,
Expand All @@ -67,6 +66,15 @@ where
Subcommand::Attach(cmd) => cmd.run(config),
Subcommand::Shutdown(cmd) => cmd.run(config).await,
Subcommand::Car(cmd) => cmd.run().await,
_ => {
let mut s = format!("{:?}", cmd).to_lowercase();
s = s.replace("(", " ");
s = s.replace(")", "");
anyhow::bail!(
"Invalid subcommand: {}. It has been moved to forest-tool binary.",
s
);
}
}
}
Err(e) => {
Expand Down
53 changes: 12 additions & 41 deletions src/cli/subcommands/db_cmd.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,25 @@

use std::sync::Arc;

use crate::cli_shared::{chain_path, cli::Config};
use crate::db::db_engine::db_root;
use crate::cli_shared::cli::Config;
use crate::rpc_api::progress_api::GetProgressType;
use crate::rpc_client::{db_ops::db_gc, progress_ops::get_progress};
use crate::utils::io::ProgressBar;
use chrono::Utc;
use clap::Subcommand;
use tracing::error;

use crate::cli::subcommands::{handle_rpc_err, prompt_confirm};
use crate::cli::subcommands::handle_rpc_err;

#[derive(Debug, Subcommand)]
pub enum DBCommands {
/// Show DB stats
Stats,
/// Run DB garbage collection
GC,
/// DB Clean up

elmattic marked this conversation as resolved.
Show resolved Hide resolved
// Those subcommands are hidden and only here to help users migrating to forest-tool
#[command(hide = true)]
Stats,
#[command(hide = true)]
Clean {
/// Answer yes to all forest-cli yes/no questions without prompting
#[arg(long)]
force: bool,
},
Expand All @@ -31,15 +30,6 @@ pub enum DBCommands {
impl DBCommands {
pub async fn run(&self, config: &Config) -> anyhow::Result<()> {
match self {
Self::Stats => {
use human_repr::HumanCount;

let dir = db_root(&chain_path(config));
println!("Database path: {}", dir.display());
let size = fs_extra::dir::get_size(dir).unwrap_or_default();
println!("Database size: {}", size.human_count_bytes());
Ok(())
}
Self::GC => {
let start = Utc::now();

Expand Down Expand Up @@ -81,30 +71,11 @@ impl DBCommands {

Ok(())
}
Self::Clean { force } => {
let dir = chain_path(config);
if !dir.is_dir() {
println!(
"Aborted. Database path {} is not a valid directory",
dir.display()
);
return Ok(());
}
println!("Deleting {}", dir.display());
if !force && !prompt_confirm() {
println!("Aborted.");
return Ok(());
}
match fs_extra::dir::remove(&dir) {
Ok(_) => {
println!("Deleted {}", dir.display());
Ok(())
}
Err(err) => {
error!("{err}");
Ok(())
}
}
_ => {
anyhow::bail!(
"Invalid subcommand: db {}. It has been moved to forest-tool binary.",
format!("{:?}", self).to_lowercase()
);
}
}
}
Expand Down
16 changes: 11 additions & 5 deletions src/cli/subcommands/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ mod attach_cmd;
mod auth_cmd;
mod car_cmd;
mod chain_cmd;
mod config_cmd;
mod db_cmd;
mod fetch_params_cmd;
mod info_cmd;
Expand All @@ -38,7 +37,7 @@ use tracing::error;

pub(super) use self::{
archive_cmd::ArchiveCommands, attach_cmd::AttachCommand, auth_cmd::AuthCommands,
car_cmd::CarCommands, chain_cmd::ChainCommands, config_cmd::ConfigCommands, db_cmd::DBCommands,
car_cmd::CarCommands, chain_cmd::ChainCommands, db_cmd::DBCommands,
fetch_params_cmd::FetchCommands, mpool_cmd::MpoolCommands, net_cmd::NetCommands,
send_cmd::SendCommand, shutdown_cmd::ShutdownCommand, snapshot_cmd::SnapshotCommands,
state_cmd::StateCommands, sync_cmd::SyncCommands, wallet_cmd::WalletCommands,
Expand All @@ -56,8 +55,14 @@ pub struct Cli {
pub cmd: Subcommand,
}

// This subcommand is hidden and only here to help users migrating to forest-tool
#[derive(clap::Subcommand, Debug)]
pub enum ConfigCommands {
Dump,
}

/// Forest binary sub-commands available.
#[derive(clap::Subcommand)]
#[derive(clap::Subcommand, Debug)]
pub enum Subcommand {
/// Download parameters for generating and verifying proofs for given size
#[command(name = "fetch-params")]
Expand Down Expand Up @@ -91,7 +96,8 @@ pub enum Subcommand {
#[command(subcommand)]
State(StateCommands),

/// Manage node configuration
// This subcommand is hidden and only here to help users migrating to forest-tool
#[command(hide = true)]
#[command(subcommand)]
Config(ConfigCommands),

Expand Down Expand Up @@ -203,7 +209,7 @@ pub(super) fn print_stdout(out: String) {
.unwrap();
}

fn prompt_confirm() -> bool {
pub fn prompt_confirm() -> bool {
elmattic marked this conversation as resolved.
Show resolved Hide resolved
print!("Do you want to continue? [y/n] ");
std::io::stdout().flush().unwrap();
let mut line = String::new();
Expand Down
2 changes: 1 addition & 1 deletion src/cli_shared/cli/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ impl ConfigPath {
}
}

fn find_config_path(opts: &CliOpts) -> Option<ConfigPath> {
pub fn find_config_path(opts: &CliOpts) -> Option<ConfigPath> {
if let Some(s) = &opts.config {
return Some(ConfigPath::Cli(PathBuf::from(s)));
}
Expand Down
25 changes: 24 additions & 1 deletion src/tool/main.rs
Original file line number Diff line number Diff line change
@@ -1,27 +1,50 @@
// Copyright 2019-2023 ChainSafe Systems
// SPDX-License-Identifier: Apache-2.0, MIT

use crate::cli_shared::cli::*;

use std::ffi::OsString;

use super::subcommands::Cli;
use clap::Parser;

use crate::utils::{io::read_file_to_string, io::read_toml};

use super::subcommands::Subcommand;

fn read_config() -> anyhow::Result<Config> {
let opts = CliOpts::default();
let path = find_config_path(&opts);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no CliOpts here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. I can create a small util find_config_path that doesn't require opts, sounds ok?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can you find non-default configuration paths without taking any arguments?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, this is missing for now but will handle that case.

let cfg: Config = match &path {
Some(path) => {
// Read from config file
let toml = read_file_to_string(path.to_path_buf())?;
// Parse and return the configuration file
read_toml(&toml)?
}
None => Config::default(),
};
Ok(cfg)
}

pub fn main<ArgT>(args: impl IntoIterator<Item = ArgT>) -> anyhow::Result<()>
where
ArgT: Into<OsString> + Clone,
{
// Capture Cli inputs
let Cli { cmd } = Cli::parse_from(args);

let config = read_config()?;
lemmih marked this conversation as resolved.
Show resolved Hide resolved

tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()?
.block_on(async {
// Run command
match cmd {
Subcommand::Benchmark(benchmark) => benchmark.run().await,
Subcommand::Config(cmd) => cmd.run(&config, &mut std::io::stdout()),
Subcommand::Benchmark(cmd) => cmd.run().await,
Subcommand::DB(cmd) => cmd.run().await,
}
})
}
File renamed without changes.
74 changes: 74 additions & 0 deletions src/tool/subcommands/db_cmd.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
// Copyright 2019-2023 ChainSafe Systems
// SPDX-License-Identifier: Apache-2.0, MIT

use crate::db::db_engine::db_root;
use clap::Subcommand;
use std::path::PathBuf;
use tracing::error;

use crate::cli::subcommands::prompt_confirm;

#[derive(Debug, Subcommand)]
pub enum DBCommands {
/// Show DB stats
Stats {
#[arg(long)]
chain_path: PathBuf,
},
/// DB Clean up
Clean {
/// Answer yes to all forest-cli yes/no questions without prompting
#[arg(long)]
force: bool,
#[arg(long)]
chain_path: PathBuf,
},
}

impl DBCommands {
pub async fn run(&self) -> anyhow::Result<()> {
match self {
Self::Stats { chain_path } => {
use human_repr::HumanCount;

let dir = db_root(chain_path);
if !dir.is_dir() {
println!(
"Aborted. Database path {} is not a valid directory",
dir.display()
);
return Ok(());
}
println!("Database path: {}", dir.display());
let size = fs_extra::dir::get_size(dir).unwrap_or_default();
println!("Database size: {}", size.human_count_bytes());
Ok(())
}
Self::Clean { force, chain_path } => {
let dir = chain_path;
if !dir.is_dir() {
println!(
"Aborted. Database path {} is not a valid directory",
dir.display()
);
return Ok(());
}
println!("Deleting {}", dir.display());
if !force && !prompt_confirm() {
println!("Aborted.");
return Ok(());
}
match fs_extra::dir::remove(dir) {
Ok(_) => {
println!("Deleted {}", dir.display());
Ok(())
}
Err(err) => {
error!("{err}");
Ok(())
}
}
}
}
}
}
9 changes: 9 additions & 0 deletions src/tool/subcommands/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
// SPDX-License-Identifier: Apache-2.0, MIT

pub mod benchmark_cmd;
pub mod config_cmd;
pub mod db_cmd;

use crate::cli_shared::cli::HELP_MESSAGE;
use crate::utils::version::FOREST_VERSION_STRING;
Expand All @@ -19,7 +21,14 @@ pub struct Cli {
/// forest-tool sub-commands
#[derive(clap::Subcommand)]
pub enum Subcommand {
/// Manage node configuration
#[command(subcommand)]
Config(config_cmd::ConfigCommands),

/// Benchmark various Forest subsystems
#[command(subcommand)]
Benchmark(benchmark_cmd::BenchmarkCommands),
/// Database management
#[command(subcommand)]
DB(db_cmd::DBCommands),
}