Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(aspects): priority-ordered aspect invocation #32097

Merged
merged 76 commits into from
Nov 29, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
9b9ce2f
WIP - initial Aspect work; added AspectApplication class, initial uni…
sumupitchayan Nov 11, 2024
6e6cb16
WIP - initial pass rewriting invokeAspects function, yet to be tested
sumupitchayan Nov 11, 2024
76c923b
fix build errors
sumupitchayan Nov 11, 2024
ec1d7b0
Merge remote-tracking branch 'origin/main' into sumughan/priority-ord…
sumupitchayan Nov 11, 2024
c88f355
add docstrings to AspectApplication class
sumupitchayan Nov 11, 2024
af5522b
WIP - add unit test for in-place mutation Aspect getting applied
sumupitchayan Nov 11, 2024
e8388eb
WIP - add unit test for single mutating aspect that adds a node
sumupitchayan Nov 11, 2024
3a7d05a
WIP - add 2 mutating aspects test (currently failing)
sumupitchayan Nov 11, 2024
70a437d
WIP - fix 2 mutating Aspect test
sumupitchayan Nov 12, 2024
12d54f9
WIP - remove duplicated aspects array in Aspects class
sumupitchayan Nov 12, 2024
6123866
WIP - return copied list in list() function
sumupitchayan Nov 12, 2024
5e016f6
WIP - wrap priority in AspectOptions for future extensibility
sumupitchayan Nov 12, 2024
549fbbd
WIP/refactor - make priority setter public, remove function
sumupitchayan Nov 12, 2024
7e91acb
WIP - add AspectPriority class with static default variables
sumupitchayan Nov 12, 2024
8f208b1
docstrings for AspectPriority static variables
sumupitchayan Nov 12, 2024
40f2a82
WIP - add check to prevent crossing stage boundary, fixing stage tests
sumupitchayan Nov 12, 2024
0786786
delete commented out old aspects function
sumupitchayan Nov 13, 2024
477deaa
WIP - add test using Tags Aspect
sumupitchayan Nov 13, 2024
9706de9
WIP - add warning for nested aspects, fixes unit test
sumupitchayan Nov 13, 2024
2341245
linter errors
sumupitchayan Nov 14, 2024
99910e9
WIP - fix orderedPriority array in invokeAspects to order numbers, no…
sumupitchayan Nov 18, 2024
09f8643
WIP - use PriorityQueue in invokeAspects algo; still failing 1 unit test
sumupitchayan Nov 19, 2024
f4816be
WIP - temporarily comment out assertion in nested Aspect warning test
sumupitchayan Nov 19, 2024
8141a0e
Merge branch 'main' into sumughan/priority-ordered-aspects
sumupitchayan Nov 19, 2024
a1c9642
WIP - fix example-construct-library/test/integ.example-resource.ts
sumupitchayan Nov 19, 2024
f905bb8
Merge branch 'sumughan/priority-ordered-aspects' of https://github.co…
sumupitchayan Nov 19, 2024
5f7fd33
WIP - remove console statements, try to fix broken integ tests in alpha
sumupitchayan Nov 20, 2024
dad0462
Revert "WIP - fix example-construct-library/test/integ.example-resour…
sumupitchayan Nov 20, 2024
477d003
WIP - return to stabilization loop solution.
sumupitchayan Nov 21, 2024
3ad2afe
WIP - add check that aspect priority cannot be less than priority of …
sumupitchayan Nov 21, 2024
2fee57c
Start of property test suite for aspects
rix0rrr Nov 22, 2024
13f0327
Some code improvements around the proptests
rix0rrr Nov 22, 2024
6b39626
Introduce helpers to reason about priorities
rix0rrr Nov 22, 2024
27d14d5
Fix problems with mutable state sharing
rix0rrr Nov 22, 2024
2ab5c8c
Fix state sharing of visit log as well
rix0rrr Nov 22, 2024
eaa4675
Update some more docs
rix0rrr Nov 22, 2024
83bcd50
Refactor prio code a bit
rix0rrr Nov 22, 2024
7d8f829
Add another test skeleton
rix0rrr Nov 22, 2024
9a6a8c1
WIP - add Feature Flag for aspectStabilization
sumupitchayan Nov 25, 2024
3c08fa1
WIP - fix imports in synthesis
sumupitchayan Nov 25, 2024
4254865
WIP - update README
sumupitchayan Nov 25, 2024
063acd2
Fix import order
sumupitchayan Nov 25, 2024
c650bd0
WIP - add aspect stabilization to README
sumupitchayan Nov 25, 2024
ee9a93c
Fix typos
rix0rrr Nov 25, 2024
dafc3aa
Merge branch 'sumughan/priority-ordered-aspects' of github.com:aws/aw…
rix0rrr Nov 25, 2024
e2cb8d2
Record additions of constructs and aspects
rix0rrr Nov 25, 2024
018cc41
WIP - fix duplicate invocation bug
sumupitchayan Nov 25, 2024
320d664
return false if feature flag/stabilize aspects option is undefined
sumupitchayan Nov 25, 2024
1df18e3
WIP - add back error for lower prio aspect being invoked after higher…
sumupitchayan Nov 25, 2024
aaef83f
WIP - add boolean param to afterSynth, use fc.boolean in prop tests f…
sumupitchayan Nov 25, 2024
184bd70
linter
sumupitchayan Nov 25, 2024
67398c3
Merge branch 'sumughan/priority-ordered-aspects' of https://github.co…
sumupitchayan Nov 25, 2024
6d0dd20
Rico review: add more explanation to aspectStabilization stage synth …
sumupitchayan Nov 25, 2024
81d458e
Rico doc/error comments
sumupitchayan Nov 25, 2024
ff1df34
linter: remove IAspect import from synthesis.ts
sumupitchayan Nov 25, 2024
d3409da
WIP - add 2 prop tests (currently commenting out AddingAspect)
sumupitchayan Nov 25, 2024
e4ef96c
WIP - fix test error expected string msg
sumupitchayan Nov 26, 2024
169a768
WIP - last prop test (not succeding yet tho)
sumupitchayan Nov 26, 2024
1e325dd
Don't use JSON.stringify
rix0rrr Nov 26, 2024
8161bfc
Don't keep references to constructs, always use paths
rix0rrr Nov 26, 2024
88dd0a9
Change toString to be more readable
rix0rrr Nov 26, 2024
b4f80bb
WIP - add infinite recursion unit test
sumupitchayan Nov 26, 2024
8694931
WIP - finish prop tests
sumupitchayan Nov 27, 2024
fd47e1d
Merge branch 'main' into sumughan/priority-ordered-aspects
sumupitchayan Nov 27, 2024
654c429
WIP - add feature flag version/default for future versions
sumupitchayan Nov 29, 2024
cee54b8
WIP - Rico comments on README
sumupitchayan Nov 29, 2024
42f6de2
WIP - Rico comment, remove Aspects with Third-Party Constructs sectio…
sumupitchayan Nov 29, 2024
ff7263b
WIP - Rico comment: improve error message for Aspect with lower prior…
sumupitchayan Nov 29, 2024
76c8bc4
WIP - remove unnecessary getAllConstructInScope function
sumupitchayan Nov 29, 2024
0f0b59c
Update packages/aws-cdk-lib/core/test/aspect.prop.test.ts
sumupitchayan Nov 29, 2024
0dda439
WIP - rename list to applied, also fix Warn unit test (since feature …
sumupitchayan Nov 29, 2024
f8e7d95
Merge branch 'sumughan/priority-ordered-aspects' of https://github.co…
sumupitchayan Nov 29, 2024
23c0f49
WIP - fix feature flag test
sumupitchayan Nov 29, 2024
3e62ef7
Merge branch 'main' into sumughan/priority-ordered-aspects
sumupitchayan Nov 29, 2024
c156482
WIP - fix features test (use 'V2Next' placeholder in feature flag list
sumupitchayan Nov 29, 2024
ed93f22
Merge branch 'sumughan/priority-ordered-aspects' of https://github.co…
sumupitchayan Nov 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 71 additions & 4 deletions packages/aws-cdk-lib/core/lib/aspect.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ export class Aspects {
public static of(scope: IConstruct): Aspects {
let aspects = (scope as any)[ASPECTS_SYMBOL];
if (!aspects) {
aspects = new Aspects();
aspects = new Aspects(scope);

Object.defineProperty(scope, ASPECTS_SYMBOL, {
value: aspects,
Expand All @@ -35,18 +35,23 @@ export class Aspects {
return aspects;
}

private readonly _scope: IConstruct;
private readonly _aspects: IAspect[];
private readonly _appliedAspects: AspectApplication[];
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved

private constructor() {
private constructor(scope: IConstruct) {
this._aspects = [];
this._appliedAspects = [];
this._scope = scope;
}

/**
* Adds an aspect to apply this scope before synthesis.
* @param aspect The aspect to add.
*/
public add(aspect: IAspect) {
public add(aspect: IAspect, priority?: number) {
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved
this._aspects.push(aspect);
this._appliedAspects.push(new AspectApplication(this._scope, aspect, priority ?? 600));
}

/**
Expand All @@ -55,4 +60,66 @@ export class Aspects {
public get all(): IAspect[] {
return [...this._aspects];
}
}

/**
* The list of aspects with priority which were directly applied on this scope.
*
* Also returns inherited Aspects of this node.
*/
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of list() vs allV2() ? I guess this returns a different type now so maybe list() is the move

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like all, because it lies. It's not actually "all", it's just "applied here".

Honestly, applied() might be better. I'll leave this to your judgement.

public get list(): AspectApplication[] {
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved
return this._appliedAspects;
}
}

/**
* Object respresenting an Aspect application. Stores the Aspect, the pointer to the construct it was applied
* to, and the priority value of that Aspect.
*/
export class AspectApplication {
/**
* The construct that the Aspect was applied to.
*/
public readonly construct: IConstruct;

/**
* The Aspect that was applied.
*/
public readonly aspect: IAspect;

/**
* The priority value of this Aspect. Must be non-negative integer.
*/
private _priority: number;

/**
* Initializes AspectApplication object
*/
public constructor(construct: IConstruct, aspect: IAspect, priority: number) {
this.construct = construct;
this.aspect = aspect;
this._priority = priority;
}

/**
* Gets the priority value.
*/
public get priority(): number {
return this._priority;
}

// Setter for priority
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved
private set priority(priority: number) {
if (priority < 0) {
throw new Error('Priority must be a non-negative number');
}
this._priority = priority;
}

/**
* Method that will change the priority of this AspectApplication
* @param priority - non-negative integer value
*/
public changePriority(priority: number): void {
this.priority = priority;
}
}
111 changes: 81 additions & 30 deletions packages/aws-cdk-lib/core/lib/private/synthesis.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,8 @@ import { prepareApp } from './prepare-app';
import { TreeMetadata } from './tree-metadata';
import { CloudAssembly } from '../../../cx-api';
import * as cxapi from '../../../cx-api';
import { Annotations } from '../annotations';
import { App } from '../app';
import { Aspects, IAspect } from '../aspect';
import { AspectApplication, Aspects, IAspect } from '../aspect';
import { FileSystem } from '../fs';
import { Stack } from '../stack';
import { ISynthesisSession } from '../stack-synthesizers/types';
Expand Down Expand Up @@ -214,44 +213,96 @@ function synthNestedAssemblies(root: IConstruct, options: StageSynthesisOptions)
* Aspects are not propagated across Assembly boundaries. The same Aspect will not be invoked
* twice for the same construct.
*/
// function invokeAspects(root: IConstruct) {
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved
// const invokedByPath: { [nodePath: string]: IAspect[] } = { };

// let nestedAspectWarning = false;
// recurse(root, []);

// function recurse(construct: IConstruct, inheritedAspects: IAspect[]) {
// const node = construct.node;
// const aspects = Aspects.of(construct);
// const allAspectsHere = [...inheritedAspects ?? [], ...aspects.all];
// const nodeAspectsCount = aspects.all.length;
// for (const aspect of allAspectsHere) {
// let invoked = invokedByPath[node.path];
// if (!invoked) {
// invoked = invokedByPath[node.path] = [];
// }

// if (invoked.includes(aspect)) { continue; }

// aspect.visit(construct);

// // if an aspect was added to the node while invoking another aspect it will not be invoked, emit a warning
// // the `nestedAspectWarning` flag is used to prevent the warning from being emitted for every child
// if (!nestedAspectWarning && nodeAspectsCount !== aspects.all.length) {
// Annotations.of(construct).addWarningV2('@aws-cdk/core:ignoredAspect', 'We detected an Aspect was added via another Aspect, and will not be applied');
// nestedAspectWarning = true;
// }

// // mark as invoked for this node
// invoked.push(aspect);
// }

// for (const child of construct.node.children) {
// if (!Stage.isStage(child)) {
// recurse(child, allAspectsHere);
// }
// }
// }
// }
function invokeAspects(root: IConstruct) {
const invokedByPath: { [nodePath: string]: IAspect[] } = { };

let nestedAspectWarning = false;
recurse(root, []);

function recurse(construct: IConstruct, inheritedAspects: IAspect[]) {
const node = construct.node;
const aspects = Aspects.of(construct);
const allAspectsHere = [...inheritedAspects ?? [], ...aspects.all];
const nodeAspectsCount = aspects.all.length;
for (const aspect of allAspectsHere) {
let invoked = invokedByPath[node.path];
if (!invoked) {
invoked = invokedByPath[node.path] = [];
}
const aspectsMap = collectAllAspectApplications(root);

const orderedPriorities = Array.from(aspectsMap.keys()).sort();

if (invoked.includes(aspect)) { continue; }
for (const priority of orderedPriorities) {
for (const aspectApplication of aspectsMap.get(priority)!) {
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved
invokeAspect(aspectApplication.construct, aspectApplication.aspect);
}
}
}

aspect.visit(construct);
/**
* Invokes an individual Aspect on a construct and all of its children in the construct tree.
*/
function invokeAspect(construct: IConstruct, aspect: IAspect) {
aspect.visit(construct);
for (const child of construct.node.children) {
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved
invokeAspect(child, aspect);
}
}

// if an aspect was added to the node while invoking another aspect it will not be invoked, emit a warning
// the `nestedAspectWarning` flag is used to prevent the warning from being emitted for every child
if (!nestedAspectWarning && nodeAspectsCount !== aspects.all.length) {
Annotations.of(construct).addWarningV2('@aws-cdk/core:ignoredAspect', 'We detected an Aspect was added via another Aspect, and will not be applied');
nestedAspectWarning = true;
/**
* Returns a Map of AspectApplications organized by Priority value.
*
* Returs a Map where the keys are Priority (number) and the values are Set<AspectApplication>
*/
function collectAllAspectApplications(root: IConstruct): Map<number, Set<AspectApplication>> {
let aspectsMap = new Map<number, Set<AspectApplication>>();

function aspectsFromNode(node: IConstruct) {
for (const aspectApplication of Aspects.of(node).list) {
const curPriority = aspectApplication.priority;
if (!aspectsMap.has(curPriority)) {
aspectsMap.set(curPriority, new Set());
}

// mark as invoked for this node
invoked.push(aspect);
let aspectsSet = aspectsMap.get(curPriority)!;
aspectsSet.add(aspectApplication);

aspectsMap.set(curPriority, aspectsSet);
}

for (const child of construct.node.children) {
if (!Stage.isStage(child)) {
recurse(child, allAspectsHere);
}
for (const child of node.node.children) {
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved
aspectsFromNode(child);
}
}

aspectsFromNode(root);

return aspectsMap;
sumupitchayan marked this conversation as resolved.
Show resolved Hide resolved
}

/**
Expand Down
116 changes: 115 additions & 1 deletion packages/aws-cdk-lib/core/test/aspect.test.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import { Construct, IConstruct } from 'constructs';
import * as cxschema from '../../cloud-assembly-schema';
import { App } from '../lib';
import { App, CfnResource, Stack } from '../lib';
import { IAspect, Aspects } from '../lib/aspect';
import { Bucket, CfnBucket } from '../../aws-s3';
import { Template } from '../../assertions';

class MyConstruct extends Construct {
public static IsMyConstruct(x: any): x is MyConstruct {
Expand All @@ -24,6 +26,29 @@ class MyAspect implements IAspect {
}
}

class EnableBucketVersioningAspect implements IAspect {
public visit(node: IConstruct): void {
if (node instanceof CfnBucket) {
node.versioningConfiguration = {
status: 'Enabled'
};
}
}
}

class AddLoggingBucketAspect implements IAspect {
public visit(node: IConstruct): void {
// Check if the node is an S3 Bucket
if (node instanceof CfnBucket) {
// Add a new logging bucket Bucket to the stack this bucket belongs to.
const stack = Stack.of(node);
new Bucket(stack, 'my-new-logging-bucket-from-aspect', {
bucketName: 'my-new-logging-bucket-from-aspect'
});
}
}
}

describe('aspect', () => {
test('Aspects are invoked only once', () => {
const app = new App();
Expand Down Expand Up @@ -69,4 +94,93 @@ describe('aspect', () => {
expect(root.node.metadata.length).toEqual(1);
expect(child.node.metadata.length).toEqual(1);
});

test('Aspects applied without priority get the default priority value', () => {
const app = new App();
const root = new MyConstruct(app, 'Construct');
const child = new MyConstruct(root, 'ChildConstruct');

// WHEN - adding an Aspect without priority specified
Aspects.of(root).add(new MyAspect());

// THEN - the priority is set to default (600)
let aspectApplication = Aspects.of(root).list[0];
expect(aspectApplication.priority).toEqual(600);
});

test('Can override Aspect priority with changePriority() function', () => {
const app = new App();
const root = new MyConstruct(app, 'Construct');
const child = new MyConstruct(root, 'ChildConstruct');

// WHEN - adding an Aspect without priority specified and resetting it.
Aspects.of(root).add(new MyAspect());
let aspectApplication = Aspects.of(root).list[0];

// THEN - we can reset the priority of an Aspect
aspectApplication.changePriority(0);
expect(Aspects.of(root).list[0].priority).toEqual(0);
});

test('In-place mutating Aspect gets applied', () => {
const app = new App();
const stack = new Stack(app, 'My-Stack');

// GIVEN - Bucket with versioning disabled
const bucket = new Bucket(stack, 'my-bucket', {
versioned: false,
});

// WHEN - adding the Aspect to enable bucket versioning gets applied:
Aspects.of(stack).add(new EnableBucketVersioningAspect());

// THEN - Aspect is successfully applied
Template.fromStack(stack).hasResourceProperties('AWS::S3::Bucket', {
'VersioningConfiguration': {
'Status': 'Enabled',
}
});
});

test('mutating Aspect that creates a node gets applied', () => {
const app = new App();
const stack = new Stack(app, 'My-Stack');

// GIVEN - Bucket with versioning disabled
const bucket = new Bucket(stack, 'my-bucket', {
bucketName: 'my-original-bucket',
versioned: false,
});

// WHEN - adding the Aspect to add a logging bucket:
Aspects.of(stack).add(new AddLoggingBucketAspect());

// THEN - Aspect is successfully applied, new logging bucket is added
Template.fromStack(stack).hasResourceProperties('AWS::S3::Bucket', {
'BucketName': 'my-new-logging-bucket-from-aspect'
});
});

test('can set mutating Aspects in specified orcder and visit newly created node', () => {
const app = new App();
const stack = new Stack(app, 'My-Stack');

// GIVEN - Bucket with versioning disabled
const bucket = new Bucket(stack, 'my-bucket', {
bucketName: 'my-original-bucket',
versioned: false,
});

// WHEN - adding both Aspects but making LoggingBucket Aspect run first
Aspects.of(stack).add(new AddLoggingBucketAspect(), 0);
Aspects.of(stack).add(new EnableBucketVersioningAspect());

// THEN - both Aspects are successfully applied, new logging bucket is added with versioning enabled
Template.fromStack(stack).hasResourceProperties('AWS::S3::Bucket', {
'BucketName': 'my-new-logging-bucket-from-aspect',
'VersioningConfiguration': {
'Status': 'Enabled',
}
});
});
});
Loading