(cli): Ignore stacks from diff and deploy

[nomike]

Describe the feature

Add a flag to the CLI to allow not deploying certain stacks.

Use Case

I maintain a large code-base which contains 669 stacks.
There currently is an issue with two of those stacks, which fail during deployment. A fix for those issues is complex and entails some risk. Due to the upcoming holiday season it was decided to postpone the fixing activities to mid of January.

So right now, I could not run a full cdk deploy as those two stacks would fail.
Those two stacks consume exports from other stacks. So if I were to temporarily remove them from the code-base, CDK would attempt to remove the exports from that other two stacks, causing them to fail.

These are automatically generated exports due to cross-region resource sharing. So adding exports manually to those other stacks doesn't really work. Those exports contain the names of the failing stacks in their IDs, so I also couldn't just create a dummy stack consuming those to keep those exports.

So currently. the only thing I can do is run cdk deploy with a list of all stacks, except the two broken ones. During compile-time, the broken stacks will still be there, so the exports will stay in place, they will just not be deployed.

This however is cumbersome and I would rather deploy all stacks as usual and exclude the broken ones.

Proposed Solution

An option --exclude-stack=<stackname> should be added to both cdk diff and cdk deploy which just skips those stacks.
The option could be specified multiple times if you want to exclude more than one stack.

If the --all option is used, the behavior is clearly defined.

If however, a list of stacks to be deployed is provided, I would say this flag should override that. This would allow things like:

find -name prod-*.json | sed 's/\.json//' | xargs cdk deploy --exclude prod-broken-stack

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.172.0 (build 0f666c5)

Environment details (OS name and version, etc.)

Does not apply

[nmussy]

I like the idea of being able to exclude application stacks, or even include them via their deployment status, e.g. running a cdk destroy only on ROLLBACK_FAILED stacks.

The one issue I can see with excluding stacks that included ones are dependent on. There is already an --exclusive flag that allows to override this dependency inclusion, but not using it with an exclusion filter could have an excluded stack be included. Given the following app:

const app = new App();
const stackA = new TestStack(app, "StackA", {});
const stackB = new TestStack(app, "StackB", {});

stackB.addDependency(stackA);

const stackC = new TestStack(app, "StackC", {});

The current behavior is as follows:

# Deploys stacks A, B and C
cdk deploy --all
# Deploys stacks A and B
cdk deploy StackB
# Only deploys stack B
cdk deploy StackB --exclusively

Stacks being implicitly included here is not a big deal, because the user requested that a given stack be deployed. Dependent stacks being included do not go against their explicit request, and they can override it with --exclusively.

With a new exclusion flag, we might end up with excluded stacks still being deployed:

# Only deploys stack B
cdk deploy StackB  --exclusively --exclude StackA,StackC
# Deploys stacks B and C, A is excluded because we used the --exclusively flag?
cdk deploy  --all --exclusively --exclude StackA
# Deploys stacks B and A, A is included because we didn't use the --exclusively flag?
cdk deploy StackB --exclude StackA

[khushail]

Hi @nomike , thanks for requesting this feature. @nmussy , thanks for sharing your insights and analysis.

IMO, it makes sense to have an option to just ignore stacks which does not need to be deployed when there is a whole bundle of stacks. Although CDK has exclusive flag option but that is only for the exclusive deployment and does not serve the same purpose as feature here. I also agree with @nmussy that one must consider the scenario of dependent stacks and how the command options would work in different cases.

However this seems useful to have, I am marking this feature as P2 which means it won't be immediately addressed by the team but would be open for contribution by the community.

Also I would request for community upvotes on this feature for making it to prioritized list of features if found useful.

Thanks

[nomike]

@nmussy I was thinking about that as well.

IMHO there are three ways to deal with this:

1. Issue a warning that some excluded stack has to be deployed nonetheless, as it's a dependency of another one. And then go forward and deploy it.
2. Gracefully fail with an error, stating that a depended stack could not be excluded.
3. Just deploy without that stack which worst case might lead to a failing deployment.

If A contains two resources 1 and 2 with only 2 being relevant for B but a deployment would only change 1, there wouldn't be an issue in the 3rd scenario. So there might be cases where this makes sense.

IMHO all three scenarios are valid in certain situations.

The difference between 1. and 3. could be the --exclusively flag, as you pointed out.

In a way, what I need is the exact opposite of --all --exclusively.

So for the moment I have the following workaround, to implement scenario 1:

Given there is a file excluded_stacks.list:

cdk ls | grep -Ev "^($(cat excluded_stacks.list| grep -Ev '^#' | tr '\n' '|')###)$" | xargs cdk deploy

Scenario 3 might be possible with:

cdk ls | grep -Ev "^($(cat excluded_stacks.list| grep -Ev '^#' | tr '\n' '|')###)$" | xargs cdk deploy --exclusively

And I'm sure, giving it some more thought would reveal a way to fake scenario 2 as well. But it's getting more and more ugly... :-/.