[Pipelines] Running additional actions in the default "Build" stage

[asterikx]

Possibility to run additional actions during the default Build phase. Apart from the CDK build, other builds such as website builds might be required. A website might be built and deployed along with the CDK app within the same pipeline.

Use Case

Running additional build actions in the default Build phase such as website builds.

Proposed Solution

Making the build stage a public property of pipelines.CdkPipeline so that custom actions can be added.

Let the user specify an array of buildActions instead of a single synthAction, e.g.

const synthAction = pipelines.SimpleSynthAction.standardYarnSynth({
  sourceArtifact,
  cloudAssemblyArtifact,
});

const pipeline = new pipelines.CdkPipeline(this, 'Pipeline', {
  pipelineName: 'MyAppPipeline',
  cloudAssemblyArtifact,
  sourceAction,
  buildActions: [synthAction, /* more actions */],
});

I think this covers all possible needs. Also, it is consistent with codepipeline.StageProps.actions:

aws-cdk/packages/@aws-cdk/aws-codepipeline/lib/pipeline.ts

Lines 47 to 51 in 403d657

	/**
	* The list of Actions to create this Stage with.
	* You can always add more Actions later by calling {@link IStage#addAction}.
	*/
	readonly actions?: IAction[];

Other

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[asterikx]

In my case, I have to build several subprojects - a frontend, lambdas (Golang), lambdas (TS), and email templates. The build outputs are referenced from the CDK code, i.e. using lambda.Code.fromAsset or s3deploy.BucketDeplyoment.

This means that the synth action (a CodeBuild project) needs to accept the build outputs of the subprojects builds (parallel CodeBuild projects) and copy them to the location where cdk synth would expect them. I'm not sure to what degree this process can be automated by the CDK. Still, I think that it makes sense to let users implement this themselves.

[tneely]

I was able to work around this by having my infrastructure, frontend, and lambda code all in the same package. My build command also builds the frontend + lambdas and then packages them into secondary artifacts that I can upload to S3 and pass along to the application stage.

It's not too pretty, and I still don't have a good way of passing the frontend/lambda assets to the application stage without relying on hardcoded bucket names, but it does work for the most part. The one catch I've found so far is that it doesn't trigger changes to s3deploy.BucketDeplyoment in my application stage, since CFn can't tell the asset bucket was updated. It may be possible to work around this by turning the bucket keys into unique timestamps.

code

[asterikx]

@tneely I do not completely follow.

Instead of using additional outputs and introducing a custom UploadAssets stage to deploy those outputs to S3, I would just add two s3deploy.BucketDeployments to the stack that is being deployed (this one).
CDK Pipeline already has a PublishAssets stage and takes care of uploading assets during pipeline execution.

Also, your install/build commands build process your projects in sequential order.

My intention was to speed the build phase by executing separate CodeBuild projects in parallel. For pure TypeScript projects, a single CodeBuild project might work well with Lerna/Yarn workspaces, but when you use different languages or different setup it gets complicated.
I think dedicated CodeBuild projects have other advantages too, e.g. you can clearly see which project failed, can use customized CodeBuild environments for each build, ...

[asterikx]

I have not thought through this completely.

My goal was/is to separate and parallelize builds from the synth action using separate CodeBuild projects. For the synth action to work, we must provide it with the output artifacts from the build projects and copy/move them to the location where cdk synth is expecting them.

This issue will allow the user to implement the required build logic herself, but I wonder if the CDK could/should handle this as well ... in any case, that might better be tackled in a different issue. What do you think @rix0rrr?