Don't run cdk package for every TypeScript lambda #2703
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I noticed that the cdk package step is by far the slowest part of each TypeScript lambda build. The package script does type checking, formatting checking, linting, runs the tests and synths the Cloudformation. We don't need to do all of this for every lambda, so instead run all the checks in the common job and only synth the Cloudformation in the invidual lambda jobs (which we need to upload to Riff Raff.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this change?
I noticed that the cdk package step is by far the slowest part of each TypeScript lambda build. The package script does type checking, formatting checking, linting, runs the tests and synths the Cloudformation. We don't need to do all of this for every lambda, so instead run all the checks in the common job and only synth the Cloudformation in the invidual lambda jobs (which we need to upload to Riff Raff).
This doesn't really improve the overall speed for a TypeScript build to complete as it just moves the cdk checks to the
commonjob and the main TS builds still have to wait forcommonto complete. It does mean the combined overall time for runners is lower and provides slightly faster feedback in the case of a cdk issue (tests failing for example). It also means in the case of a cdk issue only one job (common) will fail, instead of repeating the same failure in the individual TS lambda jobs.Update 14/2: I've now split the cdk checks and module checks (previously running sequentially in common) to separate jobs which run in parallel (there are no dependencies between them). Then when they're both successfully completed the individual lambda builds run. This does have an overall speedup effect, and maintains the benefit of a cdk failure happening in only one place.
How to test
Picking on the ticket-tailor-webhook build as an example.
Before:
CI-Typescript / common: 1m23sCI-Typescript / gu-cdk-build (ticket-tailor-webhook): 1m27sAfter:
CI-Typescript / common: 2m10sCI-Typescript / gu-cdk-build (ticket-tailor-webhook): 40sLatest:
In parallel:
CI-Typescript / modules: 1mCI-Typescript / cdk-checks: 1mThen:
CI-Typescript / gu-cdk-build (ticket-tailor-webhook): 37s(There's some variability in build run times, so take these with a grain of salt. But there does seem to be an overall improvement).
How can we measure success?
Have we considered potential risks?
Images
Accessibility