[assets] symlinks are always followed on publish, FollowMode has no effect

[mhart]

It seems that while FollowMode is respected while copying assets to cdk.out, it's not respected when publishing said assets to S3.

A cursory glance at the packaging code shows why this is the case:

aws-cdk/packages/cdk-assets/lib/private/archive.ts

Lines 8 to 16 in dac9bb3

	// The below options are needed to support following symlinks when building zip files:
	// - nodir: This will prevent symlinks themselves from being copied into the zip.
	// - follow: This will follow symlinks and copy the files within.
	const globOptions = {
	dot: true,
	nodir: true,
	follow: true,
	cwd: directory,
	};

Both nodir and follow are hardcoded here and so using FollowMode.NEVER has no effect: symlinks are always followed when uploading to S3.

Reproduction Steps

1. Create a layer directory that contains a file and a relative symlink to that file.

2. Create a CDK stack containing:

const layer = new lambda.LayerVersion(stack, 'Layer', {
  code: lambda.Code.fromAsset(path.join(__dirname, 'layer'), { follow: assets.FollowMode.NEVER }), // this is the default anyway
});

3. Deploy the stack.

Observe that in the asset directory in cdk.out, the symlink is preserved as is – it's a relative symlink that points to the file as it was when you created it.

Observe that in the uploaded asset however, the symlink has been followed (ie, the file is duplicated in the layer, there is no symlink anymore). You can either do this by using the layer in a Lambda, or by simply downloading the zipfile asset that was published to S3 and listing its contents.

So there appears to be no way to upload layers (or functions) and preserve symlinks – unless you manually zip them yourselves. Which means FollowMode has no effect.

Environment

• CLI Version : 1.54.0 (build c01b9b9)
• Framework Version: 1.54.0
• Node.js Version: v12.18.0
• OS : macOS
• Language (Version): JavaScript

---

This is 🐛 Bug Report

[eladb]

@jogold let's move asset zipping from the CLI to the app and combine the ability to allow users to output a .zip file from a bundling operation in order to allow them to fully customize (see this).

[github-actions]

This issue has not received any attention in 1 year. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[isker]

This is still a problem, as far as I can tell. I'm trying to deploy a single yarn workspace with its node_modules. Yarn gives workspaces a symlink to themselves (i.e. packages/foo/node_modules/ has a symlink foo pointing up a directory) so that they can resolve themselves by name. In this case the upload fails as it has infinitely recursed, resulting in ENAMETOOLONG.

I guess solutions are to manually exclude the offending symlink if you don't need it in the options, or to create the zip file yourself.

[revmischa]

I am running into this issue as well. The CDK bundle contains symlinks but the zip file uploaded to lambda contains no symlinks and appears to have followed the symlinks when zipping, regardless of SymlinkFollowMode.
This breaks trying to bundle dependencies for Nextjs in standalone output mode when using a pnpm monorepo.

[juanheyns]

I am on cdk v2.85.0 and it seems like the SymlinkFollowMode.NEVER does still not work.

I can confirm that zipping the contents of my folder is a workaround.

[t3r]

to keep this alive: this issue can still be observed in cdk v2.141.0