Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(aws-cdk): Receiving generic "Invalid URL" error when running cdk diff #33495

Closed
1 task
chrisdlim opened this issue Feb 18, 2025 · 18 comments
Closed
1 task

(aws-cdk): Receiving generic "Invalid URL" error when running cdk diff #33495

chrisdlim opened this issue Feb 18, 2025 · 18 comments
Assignees
Labels
bug This issue is a bug. cli Issues related to the CDK CLI p1 package/tools Related to AWS CDK Tools or CLI

Comments

@chrisdlim
Copy link

chrisdlim commented Feb 18, 2025

Describe the bug

We have our app deployed via CircleCI and we received the following error when attempting to run a cdk diff:

npm WARN exec The following package was not found and will be installed: cdk@2.1000.0
Invalid URL

Exited with code exit status 1

Explicitly installing cdk@2.151.0 fixed the issue. Are there dependency mismatches when using 2.1000.0?

Regression Issue

  • Select this option if this issue appears to be a regression.

Last Known Working CDK Version

2.179.0

Expected Behavior

To not receive an error about "Invalid URL". Using the --verbose flag also does not provide any additional debug logs.

Current Behavior

Received a generic "Invalid URL" error

Reproduction Steps

yarn add cdk@2.1000.0
npx cdk diff

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.1000.0

Framework Version

No response

Node.js Version

18.15.0

OS

Linux

Language

TypeScript

Language Version

No response

Other information

No response

@chrisdlim chrisdlim added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Feb 18, 2025
@github-actions github-actions bot added the package/tools Related to AWS CDK Tools or CLI label Feb 18, 2025
@mrgrain
Copy link
Contributor

mrgrain commented Feb 18, 2025

Hey @chrisdlim thanks for the report. Can you provide some details about the command that is running?

When I run npm install cdk@2.1000.0 everything works as expected. Also works when I run your reproduction steps.

PS: The main package is called aws-cdk. While we do provide the cdk wrapper, I generally encourage people to use the main package. It provides the same binary just with one less layer of indirection.

@bobbyu99
Copy link

Getting the same Invalid URL error from running npx cdk deploy. Does this relate to the invalid GH link github.com/aws/aws-cdk-cli on the NPM page? My npx cdk deploy command seems to try to pull the package from there.

@hbracy
Copy link

hbracy commented Feb 18, 2025

+1

@rix0rrr
Copy link
Contributor

rix0rrr commented Feb 18, 2025

We need some help to find out the triggering factor here. What is your node version? What is your npm version? What does your package.json look like? Do you have the CLI preinstalled or is npx installing it fresh?

My npx cdk deploy command seems to try to pull the package from there.

How are you determining that it is pulling a package? What is pulling a package?

This is confusing me, because npx and npm should download packages from https://registry.npmjs.com/, not https://github.com/.

is the common thread from everyone experiencing this problem that it occurs on CI systems? CodeBuild, CircleCI?

Does the error appear to be coming from npx or from cdk? Does it change anything if you run npx cdk --verbose ? What about npx --loglevel=silly cdk ?

@andreprawira
Copy link

andreprawira commented Feb 18, 2025

@rix0rrr

What is your node version

Running command cdk synth -c customer=$customer
--
143 | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
144 | !!                                                                            !!
145 | !!  Node 16 has reached end-of-life on 2023-09-11 and is not supported.       !!
146 | !!  Please upgrade to a supported node version as soon as possible.           !!
147 | !!                                                                            !!
148 | !!  This software is currently running on node v16.20.2.                      !!
149 | !!  As of the current release of this software, supported node releases are:  !!
150 | !!  - ^22.0.0 (Planned end-of-life: 2027-04-30)                               !!
151 | !!  - ^20.0.0 (Planned end-of-life: 2026-04-30)                               !!
152 | !!  - ^18.0.0 (Planned end-of-life: 2025-04-30)                               !!
153 | !!                                                                            !!
154 | !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
155 | (node:455) Warning: NodeDeprecationWarning: The AWS SDK for JavaScript (v3) will
156 | no longer support Node.js 16.x on January 6, 2025.
157 |  
158 | To continue receiving updates to AWS services, bug fixes, and security
159 | updates please upgrade to a supported Node.js LTS version.
160 |  
161 | More information can be found at: https://a.co/74kJMmI
162 | (Use `node --trace-warnings ...` to show where the warning was created)
163 | b'!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n'
164 | b'!!                                                                            !!\n'
165 | b'!!  Node 16 has reached end-of-life on 2023-09-11 and is not supported.       !!\n'
166 | b'!!  Please upgrade to a supported node version as soon as possible.           !!\n'
167 | b'!!                                                                            !!\n'
168 | b'!!  This software is currently running on node v16.20.2.                      !!\n'
169 | b'!!  As of the current release of this software, supported node releases are:  !!\n'
170 | b'!!  - ^22.0.0 (Planned end-of-life: 2027-04-30)                               !!\n'
171 | b'!!  - ^20.0.0 (Planned end-of-life: 2026-04-30)                               !!\n'
172 | b'!!  - ^18.0.0 (Planned end-of-life: 2025-04-30)                               !!\n'
173 | b'!!                                                                            !!\n'
174 | b'!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\n'

What is your npm version

npm install -g aws-cdk so i guess latest?

What does your package.json look like?

{
  "name": "cdk",
  "scripts": {
  },
  "dependencies": {
  },
  "release": {
    "branches": [
      "main"
    ],
    "plugins": [
      "@semantic-release/commit-analyzer",
      "@semantic-release/release-notes-generator",
      [
        "@semantic-release/exec",
        {
          "prepareCmd": "echo __version__ = \\\"${nextRelease.version}\\\" > version.py",
          "publishCmd": "echo NEW_RELEASE=${nextRelease.version} > TEMP_GITHUB_ENV"
        }
      ],
      [
        "@semantic-release/git",
        {
          "assets": [
            "version.py"
          ]
        }
      ],
      [
        "@semantic-release/github",
        {
          "successComment": ":tada: This has been included in the repo/package version ${nextRelease.version} :tada:"
        }
      ]
    ]
  },
  "devDependencies": {
    "@semantic-release/changelog": "^6.0.1",
    "@semantic-release/exec": "^6.0.3",
    "@semantic-release/git": "^10.0.1",
    "semantic-release": "^19.0.5"
  }
}

Previously we were on aws-cdk-lib==2.171.1, which was when we got the WebAssembly error, then we updated to aws-cdk-lib==2.175.1 which is when we started getting invalid url error.

@rix0rrr rix0rrr added p1 cli Issues related to the CDK CLI and removed needs-triage This issue or PR still needs to be triaged. labels Feb 18, 2025
@joshua-reddish
Copy link

joshua-reddish commented Feb 18, 2025

We are also seeing this issue, only on CI systems. Specifically I am seeing it using node 18. We were able to resolve it by pinning our aws-cdk version to 2.178.2 for now

It works just fine locally.

@kimd14
Copy link

kimd14 commented Feb 18, 2025

We are also seeing this issue, only on CI systems. Specifically I am seeing it using node 18. We were able to resolve it by pinning our aws-cdk version to 2.178.2 for now

It works just fine locally.

To expand on this a little, we generally call the toolkit through npx cdk@latest, which is working fine locally through toolkit version 2.1000.1 (build 4d3cdd5), but in our GitHub Actions CI workflows this fails with "Invalid URL".

To work around the issue for now, we've adjusted the workflows to use npx cdk@1.178.2, and now they're working fine.

While troubleshooting, we tried passing the --verbose flag into the CDK, and this is the specific point at which the error occurs:

[22:01:08] Notices refreshed
[22:01:08] Failed to store notices in the cache: Error: ENOENT: no such file or directory, open '/home/runner/.cdk/cache/notices.json'
[22:01:08] Toolkit stack: CDKToolkit
[22:01:08] Setting "CDK_DEFAULT_REGION" environment variable to us-east-1
[22:01:08] Looking up default account ID from STS
[22:01:08] Default account ID: ******
[22:01:08] Setting "CDK_DEFAULT_ACCOUNT" environment variable to ******
[22:01:08] context: {
  'aws:cdk:enable-path-metadata': true,
  'aws:cdk:enable-asset-metadata': true,
  'aws:cdk:version-reporting': true,
  'aws:cdk:bundling-stacks': [ '**' ]
}
[22:01:08] outdir: cdk.out
[22:01:08] env: {
  CDK_DEFAULT_REGION: 'us-east-1',
  CDK_DEFAULT_ACCOUNT: '******',
  CDK_OUTDIR: 'cdk.out',
  CDK_CLI_ASM_VERSION: '40.0.0',
  CDK_CLI_VERSION: '2.1000.1'
}
Warning:  aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
  use {@link Certificate } instead
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
  use {@link Certificate } instead
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
  use {@link Certificate } instead
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_cloudfront_origins.S3Origin is deprecated.
  Use `S3BucketOrigin` or `S3StaticWebsiteOrigin` instead.
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_cloudfront_origins.S3Origin is deprecated.
  Use `S3BucketOrigin` or `S3StaticWebsiteOrigin` instead.
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_cloudfront_origins.S3Origin#bind is deprecated.
  Use `S3BucketOrigin` or `S3StaticWebsiteOrigin` instead.
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_cloudfront_origins.S3Origin#bind is deprecated.
  Use `S3BucketOrigin` or `S3StaticWebsiteOrigin` instead.
  This API will be removed in the next major release.
[22:01:10] Notices refreshed
Invalid URL
Error: Process completed with exit code 1.

To complicate matters further, we're apparently only seeing the "Invalid URL" error from some workflows/runners in our enterprise and not from others, so some projects are running with npx cdk@latest just fine despite being set up similarly.

@rix0rrr
Copy link
Contributor

rix0rrr commented Feb 18, 2025

Thanks all! As far as I can tell it's again only on Node 16. Through some path, during validation of the Cloud Assembly manifest, the jsonschema library does the following call:

new URL('/undefined#/definitions/AssemblyManifest', 'thismessage::/')

Which succeeds on Node 18+, but throws an Invalid URL error on Node 16.

Haven't yet been able to determine why this validation is sometimes performed and sometimes not. It looks like we consumed an update to jsonschema; we used to be on 1.4.1 but we are now on 1.5.0, likely the culprit.

@rix0rrr
Copy link
Contributor

rix0rrr commented Feb 18, 2025

Stack trace for future research:

[00:50:39] TypeError [ERR_INVALID_URL]: Invalid URL
    at __node_internal_captureLargerStackTrace (node:internal/errors:465:5)
    at new NodeError (node:internal/errors:372:5)
    at URL.onParseError (node:internal/url:553:9)
    at new URL (node:internal/url:629:5)
    at Validator3.resolve10 [as resolve] (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:1444:20)
    at Validator3.validateSchema (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:1403:29)
    at Validator3.validate2 [as validate] (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:1365:26)
    at Function.validate (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:5420:35)
    at Function.loadManifest (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:5454:19)
    at Function.loadAssemblyManifest (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:5339:26)
    at new CloudAssembly5 (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:6542:44)
    at createAssembly (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:345063:12)
    at execProgram (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:345031:22)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.CloudExecutable.synthesizer (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:345501:34)
    at async CloudExecutable.doSynthesize (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:344791:28)
    at async CloudExecutable.synthesize (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:344784:33)
    at async CdkToolkit.selectStacksForDiff (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:338351:26)
    at async CdkToolkit.diff (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:337563:24)
    at async exec3 (/Users/huijbers/Temp/testinit/node_modules/aws-cdk/lib/index.js:345534:12)

@joshua-reddish
Copy link

joshua-reddish commented Feb 18, 2025

@rix0rrr We are definitely seeing the issue on node 18.15

Run actions/setup-node@v4
Resolved ./.nvmrc as 18.15
Attempting to download 18.15...
Acquiring 18.15.0 - x64 from https://github.com/actions/node-versions/releases/download/18.15.0-4362038354/node-18.15.0-linux-x64.tar.gz
Extracting ...
/usr/bin/tar xz --strip 1 --warning=no-unknown-keyword --overwrite -C /home/runner/_work/_temp/cf7ca078-5a00-48e2-83d8-cc61e67a3026 -f /home/runner/_work/_temp/44c62c98-0f67-481b-b4c0-169a3a2d80c3
Adding to the cache ...
Environment details
[22:01:08] env: {
  CDK_DEFAULT_REGION: 'us-east-1',
  CDK_DEFAULT_ACCOUNT: '****',
  CDK_OUTDIR: 'cdk.out',
  CDK_CLI_ASM_VERSION: '40.0.0',
  CDK_CLI_VERSION: '2.1000.1'
}
Warning:  aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
  use {@link Certificate } instead
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
  use {@link Certificate } instead
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_certificatemanager.DnsValidatedCertificate is deprecated.
  use {@link Certificate } instead
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_cloudfront_origins.S3Origin is deprecated.
  Use `S3BucketOrigin` or `S3StaticWebsiteOrigin` instead.
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_cloudfront_origins.S3Origin is deprecated.
  Use `S3BucketOrigin` or `S3StaticWebsiteOrigin` instead.
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_cloudfront_origins.S3Origin#bind is deprecated.
  Use `S3BucketOrigin` or `S3StaticWebsiteOrigin` instead.
  This API will be removed in the next major release.
Warning:  aws-cdk-lib.aws_cloudfront_origins.S3Origin#bind is deprecated.
  Use `S3BucketOrigin` or `S3StaticWebsiteOrigin` instead.
  This API will be removed in the next major release.
[22:01:10] Notices refreshed
Invalid URL
Error: Process completed with exit code 1.

@chrisdlim
Copy link
Author

chrisdlim commented Feb 19, 2025

I am confirming that it breaks on node v18.5.

Our CCI container is using node v18.5. We originally did not have cdk installed, so npm would automatically pull the latest cdk, install it, and run "cdk diff".

However, when cdk v2.1000.0 came out, it did the same process except it broke on this "Invalid URL" error.

Adding cdk v2.1000.0 as part of our dependencies suppressed the warning about needing to install cdk but we still ran into the error. The only way to get passed this error is to install an earlier version, which is the one before the splitting of cdk versions documented here: https://aws.amazon.com/blogs/opensource/aws-cdk-is-splitting-construct-library-and-cli/

@andreprawira
Copy link

andreprawira commented Feb 19, 2025

We solved the problem, so we used Image aws/codebuild/standard:7.0 and that solves the problem. We configure our pipeline using CDK Python and here is where we define the image version build_image=codebuild.LinuxBuildImage.STANDARD_7_0 Even though updating to 2.178.2 didnt solve the problem if the image was still 6_0

@rix0rrr
Copy link
Contributor

rix0rrr commented Feb 19, 2025

@rix0rrr We are definitely seeing the issue on node 18.15

That's interesting. I tested on 18.18 and naively assumed the behavior would be the same across all 18's, but maybe it's something that got fixed later in 18's life cycle.

@rix0rrr
Copy link
Contributor

rix0rrr commented Feb 19, 2025

Ah yeah it was fixed in 18.17

rix0rrr added a commit to aws/aws-cdk-cli that referenced this issue Feb 19, 2025
….17 (#61)

It looks like `jsonschema 1.5.0` contains changes that are not
compatible with Node versions below 18.17. Hold back the version to
1.4.1.

Relates to aws/aws-cdk#33495

---
By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache-2.0 license

---------

Signed-off-by: github-actions <github-actions@github.com>
Co-authored-by: github-actions <github-actions@github.com>
@rix0rrr
Copy link
Contributor

rix0rrr commented Feb 19, 2025

Probable fix is merged, running a release now

@rix0rrr
Copy link
Contributor

rix0rrr commented Feb 19, 2025

2.1000.2 is out and does not have this issue

@rix0rrr
Copy link
Contributor

rix0rrr commented Feb 19, 2025

Thanks for the help everyone! I'm going to close this one out.

@rix0rrr rix0rrr closed this as completed Feb 19, 2025
Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 19, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue is a bug. cli Issues related to the CDK CLI p1 package/tools Related to AWS CDK Tools or CLI
Projects
None yet
Development

No branches or pull requests

8 participants