(aws-cdk/aws-amplify): Add a sourceCodeProvider for S3 or Zipped Asset

[samkio]

Amplify allows the uploading of .ZIP, URL or S3 locations as source via the console but this is not accessible via the CDK.

Use Case

We are looking to have a git trunk model where changes are built and then deployed to multiple stages via a CDK CodePipeline. We have 3 Amplify applications; one for each stage. We do not have a way currently with the CDK to perform a manual ZIP/S3 upload for Amplify site generation but this is possible via the UI.

The case will be that we would package our Amplify app source code using CodeBuild in a CodePipeline step and store the output on S3. The CDK for each stage will then use this asset for deployment rather than a Git repository. This will ensure that exactly the same code is being deployed to all 3 stages at the time we wish to deploy.

Using this functionality we avoid using Amplifies auto-build on push of branch in favour of an artefact that is managed by a CodePipeline.

Proposed Solution

New sourceCodeProvider referencing a CodeBuild Asset or S3 file location.
Amplify app to deploy asset on update.

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[MrArnoldPalmer]

So this doesn't seem to be supported in CloudFormation as far as I can tell, or if it is the documentation is very unclear on what the Repository property can be. With that being said it seems like this is a feature that CDK could implement with custom resources, where the resource uploads the asset to S3 and makes CLI calls to change the Amplify's application source to this newly uploaded zip.

Since the Repository prop is not required in CFN, if the user doesn't pass it this custom resource could be responsible for deploying new versions of the application. This is a bit confusing I feel like since in the case that Repository is passed, the app gets updated without CDK knowing about it, so in one case CDK is managing the lifecycle of the frontend app assets and in the other, Amplify is. Some investigation is likely required to see if this custom resource is possible.

[x] 👋 I may be able to implement this feature request

@samkio would you be interested in trying to see if this works? Since a lot of Amplify is centered around using the Amplify CLI over CFN, it feels like we should get into the game of custom resources calling this CLI for more complex workflows at some point so that CDK users can manage their Amplify applications alongside all their other resources.

It feels like if we could support this it could greatly increase the viability of using Amplify in CDK since it gives users the option to manage their Amplify app's assets alongside all the other assets CDK may be managing for them, instead of Amplify looking to a separate repository. Making this a P1 because of that.

[samkio]

@MrArnoldPalmer thanks for taking a look at this.

That's exactly our issue; we have the CDK updating the infrastructure and Amplify handling it's own release lifecycle. Ideally we want these to be aligned where possible.

Happy to take a look further if this can be achieved with Custom Resources.

I took a look today and attempted to use Amplify's StartDeployment within an AwsCustomResource; unfortunately the sourceUrl must be publicly accessible which has made it difficult to work with S3 Assets (in a private bucket with encryption). If you have any ideas on this please let me know; below is that attempt.

const asset = new assets.Asset(this, "SampleAsset", {
    path: path.join(__dirname, "../my-app/build"),
});

const branchName = "main";
const amplifyApp = new amplify.App(this, "MyApp", {});
amplifyApp.addBranch("mainBranch", {
    branchName: branchName,
});

new AwsCustomResource(this, "AmplifyStartDeployment", {
    onUpdate: {
        service: "Amplify",
        action: "startDeployment",
        parameters: {
            appId: amplifyApp.appId,
            branchName: branchName,
            sourceUrl: asset.httpUrl,
        },
        physicalResourceId: PhysicalResourceId.of("SampleAsset"),
    },
    policy: AwsCustomResourcePolicy.fromSdkCalls({
        resources: AwsCustomResourcePolicy.ANY_RESOURCE,
    }),
});

Meanwhile I am writing a CustomResource backed by a lambda to do the deployment using CreateDeployment will see how that goes and revert back here.

[samkio]

Hi @MrArnoldPalmer

I got some time to look into this further today and was able to get it working using CreateDeployment.

Proof of concept repo is here (not prod-ready by any means):
https://github.com/samkio/aws-cdk-CustomAmplifyDeploymentResource

Stack: https://github.com/samkio/aws-cdk-CustomAmplifyDeploymentResource/blob/main/lib/amplify_cdk-stack.ts
Handler: https://github.com/samkio/aws-cdk-CustomAmplifyDeploymentResource/blob/main/lib/handler/index.py

I had hoped to get it to work using the AwsCustomResource as above but because of the public URL requirement it made it difficult to use with s3 assets. The solution above uses a CustomResource with a python lambda that downloads the S3 asset and then uses Amplify's create and start deployment APIs to upload and deploy the downloaded asset. It would need some work to make it production ready but it proves out that it is possible.

[MrArnoldPalmer]

@samkio very cool! I'm very excited that this works.

The documentation on StartDeployment is unclear to me. Does it actually require a public url, or can we give it a signed URL to allow access to a private bucket?

Just to clarify, StartDeployment can accept a URL where the assets live and deploy them from there, whereas CreateDeployment needs the asset locally available? Deploying from an S3 asset definitely seems ideal if possible.

[samkio]

@samkio very cool! I'm very excited that this works.

The documentation on StartDeployment is unclear to me. Does it actually require a public url, or can we give it a signed URL to allow access to a private bucket?

Hi @MrArnoldPalmer thanks and great suggestion regarding signed URLs!

I was able to confirm that it works with signed URLs to a private bucket. So for StartDeployment as long as Amplify can retrieve the asset that works.

Just to clarify, StartDeployment can accept a URL where the assets live and deploy them from there, whereas CreateDeployment needs the asset locally available? Deploying from an S3 asset definitely seems ideal if possible.

That's right. CreateDeployment allows for the API to upload the asset as it provides an URL to upload to. I prefer not having to download / upload so the StartDeployment with signed URL works better.

I've updated the POC repo with the latest changes:
https://github.com/samkio/aws-cdk-CustomAmplifyDeploymentResource

[MrArnoldPalmer]

That is great! We definitely should make a plan to get this into @aws-cdk/aws-amplify. I'm not sure if we have precedence for generating signed URL's elsewhere in custom resources but it doesn't see too hard to accomplish based on the PoC. It probably is also worth thinking about this from a security perspective, passing a presigned URL in the body of an encrypted request seems relatively safe but I wonder if there is something I'm not thinking about?

[samkio]

Thanks and sounds good. I think it is safe; pre-signed URLs are designed for this sort of thing. Short lived access to reduce the impact. Given we pass it to another AWS service I don't think there are issues.

The access only provides get objects on the specified resource for the default 5mins; we could reduce this further if necessary (as it is used immediately after the call).

In the POC I haven't implemented anything for DELETE event; as I don't know what this would actually do. Are there any concerns here as we cannot delete the custom resources as it's a deployment itself? Ideally we could revert back to the last known asset but I am not sure Amplify has this functionality.

[MrArnoldPalmer]

Oh that's a good callout. We may want to experiment with object versioning on the bucket where we store the assets and perhaps we could revert a version on a rollback.

[samkio]

So I took a look at how this works.

There are 3 events for custom resources:

Create - called when the resource is first created.
Update - called when the resource is updated.
Delete - called when the resource is deleted.

The POC has create and update to be the same logic to deploy and delete no-ops.

On a rollback there are 3 scenarios:
The deployment failed at create.
The deployment failed to update.
The deployment failed to delete.

For create there is no issue per-se as we haven't deployed the app in theory.
For update, CloudFormation will call update on the resource again but with the previous values automatically. This means if the asset changed it will revert back.
For delete, CloudFormation also no-ops.

Essentially if there is a failure on an update event (in this resource or another); CloudFormation will automatically issue another update with the old properties and therefore the old asset.

I tried this out by changing the asset and getting another resource to fail and can confirm that it did a deployment of the previous asset (rolledback).

So it looks like the current use works in this scenario by the virtue of CloudFormation handling a new update on rollbacks.

[samkio]

@MrArnoldPalmer just wanted to check in on this. Are you happy with the above? And if so is the next step to create a PR for this? Thanks

[MrArnoldPalmer]

yeah I believe so. Does the delete workflow work correctly as well? I guess cleaning up the assets and deleting the amplify resource would be all we need.

@skinny85 check this out and let us know if you're cool with a PR on this? aws-amplify being experimental we sould have some time to tweak the API if needed.

[skinny85]

@MrArnoldPalmer I'm more than cool, I'm ice-cold. Go ahead with a PR for this 👍.

[samkio]

Hi @MrArnoldPalmer ; PR is now ready for review #16922

With regards to delete it worked for me; the asset deletion should be handled by aws-s3-assets I believe as this is what the resource uses.

[ChenKuanSun]

@samkio As far as I know, some user may want choice that asset in S3 is deploy to Amplify by update or create.
Can you make it a property and allow rollback to last version if user choice to create option(CF level)?

But I think this is follow feature after this issue PR.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[dotsource]

how to update or overwrite the totalTimeout? in a CDK app I am using. The deployment is failing for a zip of size 30 MB