chore(cli): use CliIoHost as logger everywhere #32708

HBobertz · 2025-01-01T20:52:44Z

Related to #32346

Reason for this change

Capture all logs across the CLI as IoMessages so that when we allow customers to use the CDK Core functionality programmatically, these logs are delivered to the customers as messages and now just swallowed across the CLI.

Description of changes

Prevented access to log() and formatLogMessage(). Now the only ways to log something properly are either through the exported log functions (which is how everyone was doing it anyway), or through CliIoHost.notify() directly.

CliIoHost is now exposed as a global singleton which is currently only directly exclusively used in logging.ts but is effectively used everywhere as all logging functions inevitably call CliIoHost.notify()

All logging functions now optionally support the following input types

error(`operation failed: ${e}`)                                 // infers default error code `TOOLKIT_0000`
error('operation failed: %s', e)                                // infers default error code `TOOLKIT_0000`
error({ message: 'operation failed', code: 'SDK_0001' })        // specifies error code `SDK_0001`
error({ message: 'operation failed: %s', code: 'SDK_0001' }, e) // specifies error code `SDK_0001`

and everything is now translated into an IoMessage and calls CliIoHost.notify() from these logging functions. Nothing currently specifies any message code so it's all the generic _0000, _1000, _2000

Description of how you validated changes

added and updated unit tests

Checklist

My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

aws-cdk-automation

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

packages/aws-cdk/lib/logging.ts

codecov · 2025-01-01T21:07:08Z

Codecov Report

Attention: Patch coverage is 79.73568% with 46 lines in your changes missing coverage. Please review.

Project coverage is 81.40%. Comparing base (173a204) to head (59504c4).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #32708      +/-   ##
==========================================
- Coverage   81.41%   81.40%   -0.01%     
==========================================
  Files         223      223              
  Lines       13721    13727       +6     
  Branches     2416     2411       -5     
==========================================
+ Hits        11171    11175       +4     
- Misses       2271     2274       +3     
+ Partials      279      278       -1

Flag	Coverage Δ
suite.unit	`81.40% <79.73%> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
packages/aws-cdk	`80.74% <79.73%> (-0.02%)`	⬇️
packages/aws-cdk-lib/core	`82.10% <ø> (ø)`

mrgrain · 2025-01-13T17:54:10Z

packages/aws-cdk/lib/logging.ts

@@ -108,105 +46,239 @@ export async function withCorkedLogging<T>(block: () => Promise<T>): Promise<T>
  } finally {
    CORK_COUNTER--;
    if (CORK_COUNTER === 0) {
-      logBuffer.forEach(([stream, str]) => stream.write(str + '\n'));
+      // Process each buffered message through notify
+      for (const ioMessage of logBuffer) {


Backlog item for Corking. Because Corking is global and we cannot have that.

mrgrain · 2025-01-13T17:58:03Z

packages/aws-cdk/lib/os.ts

@@ -23,7 +23,7 @@ export async function shell(command: string[]): Promise<string> {

    // Both write to stdout and collect
    child.stdout.on('data', chunk => {
-      process.stdout.write(chunk);
+      data(chunk);


this probably no good. This used to just print chunks and now we are formatting chunks.

For now revert and make backlog item to deal with this.

Only used in cdk init apparently to do some setting up of Dotnet apps

Previously this line indiscriminately wrote data chunks from the child process into the output stream. These chunks are probably strings, although technically they can be a buffer.

The chunk then gets passed through formatMessageAndLog and log, to end up on ioHost.notify.
While this is probably safe, the logging helpers as written may or may not format chunks. Going forward, we should have a more explicit API for data chunks.

The problem is not with code calling data but with the eventually called functions no explicitly being aware of the possibility of not to be formatted chunks.

This is just to be defensive about unintentional changes.

https://github.com/orgs/aws/projects/257/views/1?pane=issue&itemId=93573012&issue=aws%7Caws-cdk%7C32902

mrgrain · 2025-01-13T18:17:25Z

packages/aws-cdk/lib/toolkit/cli-io-host.ts

+/**
+ * The current action being performed by the CLI. 'none' represents the absence of an action.
+ */
+export type IoAction = 'synth' | 'list' | 'deploy' | 'destroy' | 'none';


kaizencc · 2025-01-13T18:02:25Z

packages/aws-cdk/lib/api/cxapp/cloud-assembly.ts

 import { flatten } from '../../util';
+import * as chalk from '../../util/cdk-chalk';


it seems we've previously made the decision to import from ../../util (i.e. import from an index.ts file in the util folder). you are now doing something different for cdk-chalk. i am not opinionated on which way we do this, but i think it is important to be consistent, unless you have a compelling reason not to be.

maybe it means that cdk-chalk doesn't belong in util at all?

the resolution to this comment is either turning it into import { chalk, flatten} from '../../util', or moving chalk out of util, or providing a reason why it makes sense to import twice from ../../util in this file.

packages/aws-cdk/lib/api/deployments.ts

kaizencc · 2025-01-13T18:06:12Z

packages/aws-cdk/lib/api/logs/logs-monitor.ts

 import { flatten } from '../../util/arrays';
+import * as chalk from '../../util/cdk-chalk';


Ugh... it annoys me that we can import flatten multiple ways, from ../../utill/arrays and ../../util. Maybe the solution is to remove util/index.ts because that seems irrelevant. maybe i do this in a separate PR...

…ge-code-validation

### Reason for this change Instead of running regexp at runtime, we validate message codes at compile time ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

aws-cdk-automation · 2025-01-13T20:37:12Z

➡️ PR build request submitted to test-main-pipeline ⬅️

A maintainer must now check the pipeline and add the pr-linter/cli-integ-tested label once the pipeline succeeds.

aws-cdk-automation · 2025-01-13T21:05:32Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
Commit ID: 59504c4
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

mergify · 2025-01-14T01:50:31Z

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

github-actions · 2025-01-14T02:01:58Z

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

HBobertz added 2 commits January 1, 2025 15:48

initial functional IoHost logging

d304251

Merge branch 'main' into bobertzh/CliIoHost

12396ca

aws-cdk-automation requested a review from a team January 1, 2025 20:52

github-actions bot added the p2 label Jan 1, 2025

mergify bot added the contribution/core This is a PR that came from AWS. label Jan 1, 2025

HBobertz requested a deployment to test-pipeline January 1, 2025 20:53 — with GitHub Actions Waiting

aws-cdk-automation previously requested changes Jan 1, 2025

View reviewed changes

aws-cdk-automation added the pr/needs-cli-test-run This PR needs CLI tests run against it. label Jan 1, 2025

HBobertz commented Jan 1, 2025

View reviewed changes

packages/aws-cdk/lib/logging.ts Show resolved Hide resolved

HBobertz and others added 2 commits January 3, 2025 13:32

add testing

72c93da

Merge branch 'main' into bobertzh/CliIoHost

498dde1

HBobertz requested a deployment to test-pipeline January 3, 2025 18:33 — with GitHub Actions Waiting

fix lint problems

fa3513d

HBobertz requested a deployment to test-pipeline January 3, 2025 18:51 — with GitHub Actions Waiting

better comments

ea2f842

HBobertz requested a deployment to test-pipeline January 3, 2025 20:15 — with GitHub Actions Waiting

lint fixes

a5b59a6

HBobertz requested a deployment to test-pipeline January 3, 2025 20:50 — with GitHub Actions Waiting

update logic and improve testing

6f9add3

HBobertz requested a deployment to test-pipeline January 5, 2025 18:40 — with GitHub Actions Waiting

update non-functionals

306f08b

HBobertz requested a deployment to test-pipeline January 5, 2025 18:45 — with GitHub Actions Waiting

update log function name

879fa3c

HBobertz requested a deployment to test-pipeline January 5, 2025 18:56 — with GitHub Actions Waiting

change comment

255dc60

HBobertz requested a deployment to test-pipeline January 5, 2025 18:58 — with GitHub Actions Waiting

add action logic

cc2f7a9

HBobertz marked this pull request as ready for review January 5, 2025 19:11

HBobertz requested a review from a team as a code owner January 5, 2025 19:11

mrgrain reviewed Jan 13, 2025

View reviewed changes

kaizencc reviewed Jan 13, 2025

View reviewed changes

mrgrain added 2 commits January 13, 2025 18:46

feat(toolkit): enforce codes at compile time

8330271

fixup test

699edd4

HBobertz requested a deployment to test-pipeline January 13, 2025 19:34 — with GitHub Actions Waiting

address feedback

c9bc8ad

HBobertz force-pushed the bobertzh/CliIoHost branch from cd9d1ee to c9bc8ad Compare January 13, 2025 19:38

HBobertz requested a deployment to test-pipeline January 13, 2025 19:39 — with GitHub Actions Waiting

HBobertz and others added 3 commits January 13, 2025 14:42

Merge branch 'bobertzh/CliIoHost' into mrgrain/fix-compile-time-messa…

f342b96

…ge-code-validation

fix tests

0f2fb63

HBobertz requested a deployment to test-pipeline January 13, 2025 20:10 — with GitHub Actions Waiting

Merge branch 'main' into bobertzh/CliIoHost

59504c4

HBobertz temporarily deployed to test-pipeline January 13, 2025 20:36 — with GitHub Actions Inactive

HBobertz mentioned this pull request Jan 13, 2025

Handle shell logs being sent to IoHost #32902

Open

kaizencc self-requested a review January 13, 2025 21:06

kaizencc approved these changes Jan 13, 2025

View reviewed changes

HBobertz added pr-linter/exempt-integ-test The PR linter will not require integ test changes pr-linter/cli-integ-tested Assert that any CLI changes have been integ tested labels Jan 14, 2025

aws-cdk-automation removed the pr/needs-cli-test-run This PR needs CLI tests run against it. label Jan 14, 2025

HBobertz merged commit ef135ef into main Jan 14, 2025
27 of 32 checks passed

HBobertz deleted the bobertzh/CliIoHost branch January 14, 2025 02:01

github-actions bot locked as resolved and limited conversation to collaborators Jan 14, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(cli): use CliIoHost as logger everywhere #32708

chore(cli): use CliIoHost as logger everywhere #32708

HBobertz commented Jan 1, 2025 •

edited

Loading

aws-cdk-automation left a comment

codecov bot commented Jan 1, 2025 •

edited

Loading

mrgrain Jan 13, 2025

HBobertz Jan 13, 2025

mrgrain Jan 13, 2025

mrgrain Jan 13, 2025

mrgrain Jan 13, 2025 •

edited

Loading

mrgrain Jan 13, 2025

HBobertz Jan 13, 2025

mrgrain Jan 13, 2025

mrgrain Jan 13, 2025

kaizencc Jan 13, 2025

kaizencc Jan 13, 2025

aws-cdk-automation commented Jan 13, 2025

aws-cdk-automation commented Jan 13, 2025

mergify bot commented Jan 14, 2025

github-actions bot commented Jan 14, 2025

	export type IoAction = 'synth' \| 'list' \| 'deploy' \| 'destroy' \| 'none';
	export type ToolkitAction = 'synth' \| 'list' \| 'deploy' \| 'destroy' \| 'none';

		import { flatten } from '../../util';
		import * as chalk from '../../util/cdk-chalk';

		import { flatten } from '../../util/arrays';
		import * as chalk from '../../util/cdk-chalk';

chore(cli): use CliIoHost as logger everywhere #32708

chore(cli): use CliIoHost as logger everywhere #32708

Conversation

HBobertz commented Jan 1, 2025 • edited Loading

Related to #32346

Reason for this change

Description of changes

Description of how you validated changes

Checklist

aws-cdk-automation left a comment

Choose a reason for hiding this comment

codecov bot commented Jan 1, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrgrain Jan 13, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aws-cdk-automation commented Jan 13, 2025

aws-cdk-automation commented Jan 13, 2025

AWS CodeBuild CI Report

mergify bot commented Jan 14, 2025

github-actions bot commented Jan 14, 2025

HBobertz commented Jan 1, 2025 •

edited

Loading

codecov bot commented Jan 1, 2025 •

edited

Loading

mrgrain Jan 13, 2025 •

edited

Loading