Bug: sam build command doesn't support terraform's nested directory structure

[shazi57]

Description:

Relative paths provided in the source for modules throw errors when sam build command is run.

Steps to reproduce:

  {root}/environments/dev/main.tf

  module "proxy_integrated_lambda" {
  source = "../../modules/lambda"
}

in {root}/modules/lambda

resource "aws_lambda_function" "terraform_lambda_func" {
  function_name                  = "${var.service_name}-${var.filename}-${var.env}"
  role                           = "${var.role}"
  handler                        = "${var.filename}.handler"
  s3_bucket                      = "${var.bucket_name}"
  s3_key                         = "${var.service_name}.zip"
  s3_object_version              = "${var.version_id}"
  publish                        = true
  runtime                        = "nodejs16.x"
  environment {
    variables = var.envvars
  }
}

resource "null_resource" "sam_metadata_aws_lambda_function_terraform_lambda_func" {
    triggers = {
        resource_name = "aws_lambda_function.terraform_lambda_func"
        resource_type = "ZIP_LAMBDA_FUNCTION"
        original_source_code = "./src"
        built_output_path = "./src/${var.service_name}.zip"
    }
}

• cli command
  sam build --hook-name terraform --beta-features

Observed result:

Running Prepare Hook to prepare the current application
Executing prepare hook of hook "terraform"
Initializing Terraform application
.....
Creating terraform plan and getting JSON output
.............
Generating metadata file
Finished generating metadata file. Storing in /Users/tryu2/terraform-practice/environments/dev/.aws-sam-iacs/iacs_metadata/template.json
Prepare hook completed and metadata file generated at: /Users/tryu2/terraform-practice/environments/dev/.aws-sam-iacs/iacs_metadata/template.json
Building codeuri: /Users/tryu2/terraform-practice/environments/dev/src runtime: nodejs16.x metadata: {'SkipBuild': False, 'BuildMethod': 'makefile', 'ContextPath': '/Users/tryu2/terraform-practice/environments/dev/.aws-sam-iacs/iacs_metadata', 'WorkingDirectory': '/Users/tryu2/terraform-practice/environments/dev', 'ProjectRootDirectory': '/Users/tryu2/terraform-practice/environments/dev'} architecture: x86_64 functions: module.proxy_integrated_lambda.aws_lambda_function.terraform_lambda_func
Running CustomMakeBuilder:CopySource
Running CustomMakeBuilder:MakeBuild
Current Artifacts Directory : /Users/tryu2/terraform-practice/environments/dev/.aws-sam/build/ModuleProxyIntegratedLambdaAwsLambdaFunctionTerraformLambdaFuncA406CF29
python3 ".aws-sam-iacs/iacs_metadata/copy_terraform_built_artifacts.py" --expression "|values|root_module|child_modules|[?address==module.proxy_integrated_lambda]|resources|[?address==\"module.proxy_integrated_lambda.null_resource.sam_metadata_aws_lambda_function_terraform_lambda_func\"]|values|triggers|built_output_path" --directory "/Users/tryu2/terraform-practice/environments/dev/.aws-sam/build/ModuleProxyIntegratedLambdaAwsLambdaFunctionTerraformLambdaFuncA406CF29" --target "module.proxy_integrated_lambda.null_resource.sam_metadata_aws_lambda_function_terraform_lambda_func"
Initializing modules...
- apigateway_proxy_integration in
- event_targets in
- inbound_eventbridge in
- lambda_iam_permissions in
- lambda_s3_bucket in
- proxy_integrated_lambda in
- target_lambdas in

Build Failed
Error: CustomMakeBuilder:MakeBuild - Make Failed: ╷
│ Error: Unreadable module directory
│ 
│ Unable to evaluate directory symlink: lstat ../../modules: no such file or
│ directory

Traceback (most recent call last):
  File "/private/var/folders/ph/fg_nz3r52sq61pply5vfm9yc0000gq/T/tmpgaln0na2/.aws-sam-iacs/iacs_metadata/copy_terraform_built_artifacts.py", line 329, in <module>
    subprocess.check_call(["terraform", "init", "-reconfigure", "-input=false", "-force-copy"])
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 369, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['terraform', 'init', '-reconfigure', '-input=false', '-force-copy']' returned non-zero exit status 1.
make: *** [build-ModuleProxyIntegratedLambdaAwsLambdaFunctionTerraformLambdaFuncA406CF29] Error 1

Expected result:

Expected it to build functions without an issue because running terraform init in the root directory doesn't cause any issue.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

{
  "version": "1.73.0",
  "system": {
    "python": "3.11.2",
    "os": "macOS-12.4-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "Not available",
    "aws_cdk": "Not available",
    "terraform": "1.3.7"
  }
}

[lucashuy]

Hi, thanks for opening this issue. I can reproduce your issue where Terraform is not able to find the correct folder once it reaches our custom Makefile solution. This seems to be related to how we prepare the artifacts for SAM CLI since we copy the current directory to a scratch folder. In your case, this will end up missing the modules/lambda folder. Let me consult with some team members for some next steps.

[lucashuy]

This is currently a limitation of our Terraform integration. Our artifact script expects all project assets and resources to be within the project folder. You can see some more discussion within our Terraform discussion thread: #4553 (comment)

As a potential workaround for now, you can try moving your modules/lambda folder to be under environments/dev if it is possible for your project.

[kidpollo]

yeah this is kind of a big deal. Its very common to want to have multiple modules and different build targets in a single repo

[dk4tz]

+1 for me

[eugbyte]

Any updates?

I am getting a similar error although with a slightly different error message. aws sam is unable to detect the root directory of the project and instead defaults to C:\Users\user\AppData. I am on windows, using go 1.20.

My error message is:

Error: CustomMakeBuilder:MakeBuild - Make Failed: Copy/Unzip unsuccessful!
Traceback (most recent call last):
  File "C:\Users\<user>\AppData\Local\Temp\<tmp_folder>\.aws-sam-iacs\iacs_metadata\copy_terraform_built_artifacts.py", line 259, in find_and_copy_assets
    copytree(abs_attribute_path, directory_path)
  File "C:\Users\<user>\AppData\Local\Temp\<tmp_folder>\.aws-sam-iacs\iacs_metadata\copy_terraform_built_artifacts.py", line 194, in copytree
    for item in os.listdir(src):
NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\<user>\\AppData\\Local\\Temp\\<tmp_folder>\\bin\\subscription\\main'
make.exe[1]: *** [build-AwsLambdaFunctionFnSubscription45C9D248] Error 1
make: *** [sam] Error 1

[SuperP4rks]

Any Updates?

I agree with another thread and this thread, an existing production release would look more like the below for a standard python application.

/project
  /infrastructure
    envrionment/
        dev/
            dev.tfvars
        test/
            test.tfvars
    main.tf
    providers.tf
  /src
    ... (lambda code) ...
  /tests
    ... (lambda tests) ...

I am surprised though, that if the original_source_code is declared in the metadata that this could not simply be pulled through as a provisional requirement during the sam build process. i guess it just a thought.

resource "null_resource" "sam_metadata_..." {
  triggers = {
    original_source_code = original_source_code
  }
}

Another process could be adding it to the samconfig.toml as an include parameter.

version = 0.1
[default]
[default.build.parameters]
hook_name = "terraform"
beta_features = true

I think all things possible, but it would be good to see a more supportive approach for structured repositories and multi environment deployments. The work around for exporting TF_ARGS can work for now, but it would be nice to declare a backend and default tf_file to.

[moelasmar]

Hey,
We released a feature in latest SAM CLI release to support the Terraform project that has a nested directories structure like the following:

/project
  /infrastructure
    main_mondule
       main.tf
       providers.tf
    other_modules
        module1
        module2 
        .....
  /src
    ... (lambda code) ...

You can run sam build from the main_module directory, and add the new property --terraform-project-root-path and set its value to be a relative path to the project root directory like ./../.. or the absolute path, so the command will look like
sam build --hook-name terraform --beta-features --terraform-project-root-path ./../..

Please try the new feature, and let us know your feedback.

[moelasmar]

closing this issue, as this feature got released in SAM CLI version 1.96.0. Please open new issues if you have any new requests.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.