Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

core: crossRegionReferences generates too large buildspec in FileAsset #30326

Open
HansFalkenberg-Visma opened this issue May 24, 2024 · 6 comments
Open

core: crossRegionReferences generates too large buildspec in FileAsset #30326

HansFalkenberg-Visma opened this issue May 24, 2024 · 6 comments
Labels
@aws-cdk/assets Related to the @aws-cdk/assets package @aws-cdk/core Related to core CDK functionality bug This issue is a bug. effort/medium Medium work item – several days of effort p2

Comments

@HansFalkenberg-Visma
Copy link

HansFalkenberg-Visma commented May 24, 2024 •
edited
Loading

Describe the bug

Cross region references rely on using Lambda functions, which are deployed with the stack.

I am deploying to 17 regions with a mix of importing and exporting, so there will be 17 export writer functions and 17 readers. Per the 8 accounts I want to deploy to.

All cdk-asset deploys of the same Lambda is being done in a single FileAssetXx action. This fails because its build spec becomes too large.

The problem is somewhat lessened because -- I found -- cdk-asset will only be invoked once per environment (account:region) even if multiple stacks use it in that environment. This behavior is not obvious because a stack's ID is part of the command.

But it is still possible to exceed the limit even when deploying only to 3 accounts, if deploying to all 29 regions and having IDs that are not super short.

Expected Behavior

Cross region references in a pipeline are not limited by number of regions and accounts.

Current Behavior

InfrastructurePipeline/InfrastructurePipeline/Assets/FileAsset14 (InfrastructurePipelineAssetsFileAsset14268FCA88)
Failed to call UpdateProject, reason: Max buildspec length is 25600
(Service: AWSCodeBuild; Status Code: 400; Error Code: InvalidInputException; Request ID: ; Proxy: null)

Reproduction Steps

import { App, Stack, Stage } from 'aws-cdk-lib';
import {
  Effect,
  PolicyDocument,
  PolicyStatement,
  Role,
  ServicePrincipal,
} from 'aws-cdk-lib/aws-iam';
import { Bucket } from 'aws-cdk-lib/aws-s3';
import { CodePipeline, CodePipelineSource, ShellStep } from 'aws-cdk-lib/pipelines';

const app = new App();

const regions = [
 'af-south-1', 'ap-east-1', 'ap-northeast-1', 'ap-northeast-2', 'ap-northeast-3', 'ap-south-1', 'ap-south-2',
 'ap-southeast-1', 'ap-southeast-2', 'ap-southeast-3', 'ap-southeast-4', 'ca-central-1', 'ca-west-1',
 'eu-central-1', 'eu-central-2', 'eu-north-1', 'eu-south-1', 'eu-south-2', 'eu-west-1', 'eu-west-2', 'eu-west-3',
 'il-central-1', 'me-central-1', 'me-south-1', 'sa-east-1', 'us-east-1', 'us-east-2', 'us-west-1', 'us-west-2',
];

const accounts = ['123456789001', '123456789002', '123456789003', '123456789004'];

class MyPipeline extends Stack {
  constructor() {
    super(app, 'InfrastructurePipelineStack', {
      env: { account: '123456789099', region: 'eu-west-1' },
    });

    const pipeline = new CodePipeline(this, 'InfrastructurePipeline', {
      crossAccountKeys: true,
      synth: new ShellStep('Synth', {
        commands: [],
        input: CodePipelineSource.connection('o/r', 'b', { connectionArn: '' }),
      }),
    });

    for (const account of accounts) {
      const stage = new Stage(this, `Stage-${account}`);

      const exportingStack = new Stack(stage, `ExportingStack-${account}`, {
        env: { account: account, region: 'eu-west-1' },
        crossRegionReferences: true,
      });
      const exportedBucket = new Bucket(exportingStack, 'ExportedBucket');

      for (const region of regions) {
        const stack = new Stack(stage, `Stack-${account}-${region}`, {
          env: { account: account, region: region },
          crossRegionReferences: true,
        });
        new Role(stack, 'ImportingRole', {
          assumedBy: new ServicePrincipal('service.amazonaws.com'),
          inlinePolicies: {
            actions: new PolicyDocument({
              statements: [
                new PolicyStatement({
                  effect: Effect.ALLOW,
                  actions: ['s3:*'],
                  resources: [exportedBucket.bucketArn],
                }),
              ],
            }),
          },
        });
      }

      pipeline.addStage(stage);
    }
  }
}

new MyPipeline();

The resulting BuildSpec is about 32k. Even if only deploying to 3 accounts (a not uncommon pattern with ['test'], 'staging', 'production' and 'pipeline') it would be possible to exceed 25600 if identifiers were 20 characters longer than this example.

{
  "version": "0.2",
  "phases": {
    "install": {
      "commands": [
        "npm install -g cdk-assets@2"
      ]
    },
    "build": {
      "commands": [
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001afsouth110207D57.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-af-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apeast15583CC4C.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apnortheast1327014FC.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-northeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apnortheast2DA4F6AF5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-northeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apnortheast3F0C917D9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-northeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsouth18AF4448F.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsouth2B3F8D462.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsoutheast1228DED0C.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-southeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsoutheast296BC5077.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-southeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsoutheast3B108EDC8.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-southeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001apsoutheast44FB59181.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ap-southeast-4"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001cacentral1EB787D18.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ca-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001cawest196843836.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-ca-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eucentral12F0D5FB5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eucentral23358FDCF.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-central-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eunorth10BE03AF1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-north-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eusouth1A6A4B877.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001eusouth2C3D35869.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001euwest2A8E8BCAA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001euwest3B846D522.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-eu-west-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001ilcentral1B4F5BAA9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-il-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001mecentral12995A8D5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-me-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001mesouth1D1DDBAED.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-me-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001saeast1826DC9A5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-sa-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001useast1B40BBABB.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-us-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001useast2A7D83BF3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-us-east-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001uswest1D0A050EA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-us-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789001/InfrastructurePipelineStackStage123456789001Stack123456789001uswest2900A58F0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789001-us-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002afsouth1A9A890AD.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-af-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apeast1E2396BA9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apnortheast1358FB29B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-northeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apnortheast2E45F7D39.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-northeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apnortheast35E5D7B08.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-northeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsouth184F84FC5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsouth242E1E678.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsoutheast1868C395D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-southeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsoutheast2DB0112F0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-southeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsoutheast3FC7F7215.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-southeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002apsoutheast4D1D52865.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ap-southeast-4"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002cacentral12B53A514.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ca-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002cawest1F96F5390.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-ca-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eucentral1D6B1E887.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eucentral2DE4D33FA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-central-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eunorth17B22D63D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-north-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eusouth1F23BD9E7.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002eusouth23ECE6AD5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002euwest2F1A6BF51.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002euwest340F156C1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-eu-west-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002ilcentral16876F965.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-il-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002mecentral10F5E9510.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-me-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002mesouth176B6F4A8.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-me-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002saeast1FAD8C4AF.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-sa-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002useast1536C8BA2.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-us-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002useast211EDE12B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-us-east-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002uswest19E510473.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-us-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789002/InfrastructurePipelineStackStage123456789002Stack123456789002uswest2BF50BFEC.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789002-us-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003afsouth111648FBA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-af-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apeast163F172C8.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apnortheast16047CF18.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-northeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apnortheast2F0E7F5A0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-northeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apnortheast32ADB7D88.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-northeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsouth10AAC2BF0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsouth2D70AEAC9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsoutheast16D0FC8F0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-southeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsoutheast2FB5FCC1F.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-southeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsoutheast356FC751C.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-southeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003apsoutheast4757670D0.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ap-southeast-4"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003cacentral19B2A9A26.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ca-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003cawest1E14E083B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-ca-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eucentral19F21F5D6.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eucentral289CB64A6.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-central-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eunorth134B0DAE1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-north-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eusouth11F9E66BA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003eusouth23067775B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003euwest2D37CD1D1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003euwest350B35414.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-eu-west-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003ilcentral1C9E576C7.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-il-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003mecentral15743A928.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-me-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003mesouth1FAE1D361.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-me-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003saeast19C3B9507.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-sa-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003useast17B02A3D3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-us-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003useast238196F94.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-us-east-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003uswest117F8A7A9.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-us-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789003/InfrastructurePipelineStackStage123456789003Stack123456789003uswest23FC39B3E.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789003-us-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004afsouth1EE59763C.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-af-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apeast191D1E59B.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apnortheast1DCEF24E6.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-northeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apnortheast27F26CFE3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-northeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apnortheast349B5279D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-northeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsouth106E826A5.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsouth2A772FBA1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsoutheast1F3D6CBDF.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-southeast-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsoutheast2463FA3CD.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-southeast-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsoutheast3C9BF8705.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-southeast-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004apsoutheast4FC3AE12D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ap-southeast-4"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004cacentral150DAAD89.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ca-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004cawest12423800D.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-ca-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eucentral1D7A4CFE3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eucentral250F06848.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-central-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eunorth1296091BA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-north-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eusouth102FE31D3.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004eusouth283DE98BA.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-south-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004euwest2AB3726FD.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-west-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004euwest3689D7C9F.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-eu-west-3"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004ilcentral1BE6000B7.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-il-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004mecentral1181220A1.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-me-central-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004mesouth11C1A8FC2.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-me-south-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004saeast11C91DA1A.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-sa-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004useast140DDCAB8.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-us-east-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004useast20C681DBC.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-us-east-2"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004uswest13582BC32.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-us-west-1"",
        "cdk-assets --path "assembly-InfrastructurePipelineStack-Stage-123456789004/InfrastructurePipelineStackStage123456789004Stack123456789004uswest2C7009EF7.assets.json" --verbose publish "8acca95a9957d02a9f3ec124c9869c5d5b70a7fb3e332120850781ecc9363037:123456789004-us-west-2""
      ]
    }
  }
}

Possible Solution

Split into multiple FileAction actions if build spec becomes too large.

Additional Information/Context

No response

CDK CLI Version

2.143.0 (build 9f2bdf7)

Framework Version

No response

Node.js Version

v18.18.2

OS

Windows 10 Version 22H2

Language

TypeScript

Language Version

No response

Other information

No response
@HansFalkenberg-Visma HansFalkenberg-Visma added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels May 24, 2024
@github-actions github-actions bot added the @aws-cdk/assets Related to the @aws-cdk/assets package label May 24, 2024
@khushail khushail self-assigned this May 24, 2024
@khushail khushail added investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed needs-triage This issue or PR still needs to be triaged. labels May 24, 2024
This was referenced May 27, 2024
Closed
Closed
@nindzyago
Copy link

Faced same issue today. Any workarounds?

@HansFalkenberg-Visma
Copy link
Author

HansFalkenberg-Visma commented Jun 24, 2024 •
edited
Loading

I happened to be in the fortunate situation that is was possible to split deployment of a group of the accounts into a different pipeline. So even though I didn't really want a separate pipeline for this it wasn't a huge imposition either.

If that's not possible for you, I guess you could always manually manipulate the pipeline template and split the BuildSpec into two different actions as CDK should have done for you. If you want a starting point for that, I've done such manipulation for a totally different reason in #29592 and it has a .zip file with fully functional TypeScript code attached.

@khushail khushail added p2 effort/medium Medium work item – several days of effort @aws-cdk/core Related to core CDK functionality labels Jun 24, 2024
@khushail
Copy link
Contributor

khushail commented Jun 27, 2024 •
edited
Loading

agree with your sggested workaround @HansFalkenberg-Visma .
Marking the issue for further consideration by the team

@khushail khushail removed the investigating This issue is being investigated and/or work is in progress to resolve the issue. label Jun 27, 2024
@khushail khushail removed their assignment Jun 27, 2024
@cygnius
Copy link

cygnius commented Jul 22, 2024

This is affecting us as well.
We are deploying a basic hygiene stack to all our accounts which ensure compliance, kills zombie resources, support integration tests, etc.
We are deploying to 27 regions and 7 accounts and facing the same problem.
We are splitting the pipeline into multiple pipeline to bypass this issue for now.
Requesting the team to prioritize this.

@khushail
Copy link
Contributor

khushail commented Jul 22, 2024 •
edited
Loading

@cygnius, Since this has a workaround, it won't be possible to prioritise this and bypass other high priority issue for now. Please feel free to create an Internal SIM ticket through AWS Premium Support or TAMs ,if this issue needs IMMEDIATE prioritization from your side. Other way to get prioritisation on Github is through Upvotes. Users who are facing this issue, can upvote this issue and this should soon be considered by the team.

@klakpin
Copy link

klakpin commented Sep 20, 2024

We are deploying stack to a lot of other AWS accounts and faced the same problem. Adding next property to the pipeline CDK solved the issue

new cdkPipelines.CodePipeline(this, 'pipeline', {
            ...
            publishAssetsInParallel: false,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
@aws-cdk/assets Related to the @aws-cdk/assets package @aws-cdk/core Related to core CDK functionality bug This issue is a bug. effort/medium Medium work item – several days of effort p2
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@nindzyago @klakpin @cygnius @HansFalkenberg-Visma @khushail