[1/n][Release] Renames and configuration updates #2566

AhmedSoliman · 2025-01-28T13:01:08Z

Summary:
This renames a few configuration and command line arguments while keeping compatibility with old options. This also introduces extra documentation on those configuration keys

allow-bootstrap -> auto-provision
bootstrap-num-partitions -> default-num-partitions (aligns with default-partition-replication)
[bifrost.replicated-loglet] default_replication-property -> default_log_replication

Stack created with Sapling. Best reviewed with ReviewStack.

github-actions · 2025-01-28T13:32:06Z

Test Results

7 files ±0 7 suites ±0 4m 6s ⏱️ -8s
49 tests +2 10 ✅ - 36 0 💤 - 1 0 ❌ ±0 39 🔥 +39
174 runs - 8 74 ✅ - 105 1 💤 - 2 0 ❌ ±0 99 🔥 +99

For more details on these errors, see this check.

Results for commit cb70236. ± Comparison against base commit cbf3ea7.

♻️ This comment has been updated with latest results.

Summary: This renames a few configuration and command line arguments while keeping compatibility with old options. This also introduces extra documentation on those configuration keys - `allow-bootstrap` -> `auto-provision` - `bootstrap-num-partitions` -> `default-num-partitions` (aligns with `default-partition-replication`) - `[bifrost.replicated-loglet] default_replication-property` -> `default_log_replication`

tillrohrmann

LGTM. +1 for merging :-)

tillrohrmann · 2025-01-28T16:22:39Z

crates/types/src/config/cli_option_overrides.rs

+    /// Auto Cluster Provisioning
+    ///
+    /// If true, then this node is allowed to automatically provision as a new cluster.
+    /// This node *must* have an admin role and a new nodes configuration will be created that includes this node.


I think it is no longer required that the node that provisions the cluster runs the admin role.

tillrohrmann · 2025-01-28T16:23:55Z

crates/types/src/config/common.rs

    ///
-    /// Bootstrap is allowed by default, unless it is explicitly disabled in config files.
+    /// If true, then this node is allowed to automatically provision as a new cluster.
+    /// This node *must* have an admin role and a new nodes configuration will be created that includes this node.


Same comment as above.

tillrohrmann · 2025-01-28T16:25:07Z

crates/types/src/nodes_config.rs

@@ -51,7 +51,7 @@ pub enum Role {
    // todo switch to serializing as "metadata-server" in version 1.3
    #[serde(rename(serialize = "metadata-store"))]
    MetadataServer,
-    /// [IN DEVELOPMENT] Serves a log server for replicated loglets
+    /// [PREVIEW FEATURE] Serves a log-server for replicated loglets


[__PREVIEW FEATURE__]?

tillrohrmann · 2025-01-28T17:05:18Z

crates/types/src/config/common.rs

    ///
    /// Default: true
-    pub allow_bootstrap: bool,
+    #[serde(alias = "allow-bootstrap")]


The alias might actually result into duplicate fields when trying to read a configuration where allow-bootstrap is defined.

Ugh, I couldn't actually find a good solution to this :( without investing too much time, and I don't want us to give up on the rename.

I'm not sure if there are good ideas here :(

The way I solved it in another PR is by introducing a Shadow variant: https://github.com/restatedev/restate/pull/2571/files#diff-b07de33428bbafda6c26f429c0f36178f5ef8e6215e7a1f187c72142417ecc29R249. Not particularly proud of it, though.

pcholakov · 2025-01-29T14:00:13Z

crates/types/src/config/cli_option_overrides.rs

+    /// This can also be explicitly disabled by setting this value to false.
+    ///
+    /// Default: true
+    #[clap(long, global = true, alias = "allow_bootstrap")]


Should this not be?

Suggested change

#[clap(long, global = true, alias = "allow_bootstrap")]

#[clap(long, global = true, alias = "allow-bootstrap")]

Other compound arguments use kebab-case:

Usage: restate-server <--roles [<ROLES>...]|--node-name <NODE_NAME>|--location <LOCATION>|--force-node-id <FORCE_NODE_ID>|--cluster-name <CLUSTER_NAME>|--auto-provision <AUTO_PROVISION>|--base-dir <BASE_DIR>|--metadata-store-address <METADATA_STORE_ADDRESS>|--bind-address <BIND_ADDRESS>|--advertise-address <ADVERTISE_ADDRESS>|--default-num-partitions <DEFAULT_NUM_PARTITIONS>|--shutdown-timeout <SHUTDOWN_TIMEOUT>|--tracing-endpoint <TRACING_ENDPOINT>|--tracing-runtime-endpoint <TRACING_RUNTIME_ENDPOINT>|--tracing-services-endpoint <TRACING_SERVICES_ENDPOINT>|--tracing-json-path <TRACING_JSON_PATH>|--tracing-filter <TRACING_FILTER>|--log-filter <LOG_FILTER>|--log-format <LOG_FORMAT>|--log-disable-ansi-codes <LOG_DISABLE_ANSI_CODES>>

Yep, I already fixed it but didn't update the PR yet. I'll likely wait for @tillrohrmann's PR to be merged and use his configuration shadow type.

pcholakov · 2025-01-29T14:02:42Z

crates/types/src/config/admin.rs

+    /// Note that this value only impacts the cluster initial provisioning and will not be respected after
+    /// the cluster has been provisioned.
+    ///
+    /// For provisioned clusters, use the `restatectl` utility to update it.


Maybe?

Suggested change

/// For provisioned clusters, use the `restatectl` utility to update it.

/// To update existing clusters use the `restatectl` utility.

pcholakov · 2025-01-29T14:03:06Z

crates/types/src/config/bifrost.rs

+    ///
+    /// Note that this value only impacts the cluster initial provisioning and will not be respected after
+    /// the cluster has been provisioned.
+    /// For provisioned clusters, use the `restatectl` utility to update it.


As above:

Suggested change

/// For provisioned clusters, use the `restatectl` utility to update it.

///

/// To update existing clusters use the `restatectl` utility.

pcholakov · 2025-01-29T14:11:13Z

crates/types/src/config/cli_option_overrides.rs

-    pub auto_provision_partitions: Option<bool>,
+    /// Default: 24
+    #[clap(long, global = true, alias = "bootstrap-num-partitions")]
+    pub default_num_partitions: Option<NonZeroU64>,


This breaks in a weird way when set via an env variable:

❯ RESTATE_BOOTSTRAP_NUM_PARTITIONS=1 restate-server [RT0002] configuration loading error: duplicate field `default-num-partitions`

AhmedSoliman mentioned this pull request Jan 28, 2025

Avoid unnecessary metadata read in cluster controller #2564

Merged

AhmedSoliman force-pushed the pr2566 branch from cd101ac to 4568fbf Compare January 28, 2025 13:07

AhmedSoliman marked this pull request as ready for review January 28, 2025 13:07

AhmedSoliman requested review from tillrohrmann and muhamadazmy January 28, 2025 13:22

AhmedSoliman force-pushed the pr2566 branch from 4568fbf to 0f47059 Compare January 28, 2025 13:40

AhmedSoliman added 2 commits January 28, 2025 15:14

Avoid unnecessary metadata read in cluster controller

2336f09

AhmedSoliman force-pushed the pr2566 branch from 0f47059 to cb70236 Compare January 28, 2025 15:14

tillrohrmann approved these changes Jan 28, 2025

View reviewed changes

tillrohrmann reviewed Jan 28, 2025

View reviewed changes

AhmedSoliman mentioned this pull request Jan 28, 2025

Misc improvements to cluster_controller/logs_controller #2570

Merged

pcholakov reviewed Jan 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[1/n][Release] Renames and configuration updates #2566

[1/n][Release] Renames and configuration updates #2566

AhmedSoliman commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

tillrohrmann left a comment

tillrohrmann Jan 28, 2025

tillrohrmann Jan 28, 2025

tillrohrmann Jan 28, 2025

tillrohrmann Jan 28, 2025

AhmedSoliman Jan 29, 2025

tillrohrmann Jan 29, 2025

pcholakov Jan 29, 2025 •

edited

Loading

AhmedSoliman Jan 29, 2025

pcholakov Jan 29, 2025

pcholakov Jan 29, 2025

pcholakov Jan 29, 2025

	#[clap(long, global = true, alias = "allow_bootstrap")]
	#[clap(long, global = true, alias = "allow-bootstrap")]

	/// For provisioned clusters, use the `restatectl` utility to update it.
	/// To update existing clusters use the `restatectl` utility.

[1/n][Release] Renames and configuration updates #2566

Are you sure you want to change the base?

[1/n][Release] Renames and configuration updates #2566

Conversation

AhmedSoliman commented Jan 28, 2025 • edited Loading

github-actions bot commented Jan 28, 2025 • edited Loading

Test Results

tillrohrmann left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pcholakov Jan 29, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AhmedSoliman commented Jan 28, 2025 •

edited

Loading

github-actions bot commented Jan 28, 2025 •

edited

Loading

pcholakov Jan 29, 2025 •

edited

Loading