Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(secretsmanager): secret_full_arn not returning text after dash #33489

Open
1 task
RaulPontelloCaylent opened this issue Feb 18, 2025 · 5 comments
Open
1 task
Labels
@aws-cdk/aws-secretsmanager Related to AWS Secrets Manager bug This issue is a bug. effort/medium Medium work item – several days of effort p3

Comments

@RaulPontelloCaylent
Copy link

RaulPontelloCaylent commented Feb 18, 2025

Describe the bug

I am using Python and AWS CDK to deploy:

  • RDS Instance
  • Secret
  • DMS Instance
  • DMS Endpoints
  • DMS Replication Task

The secret is used in the RDS Instance.

My DMS Source Endpoint is the RDS Instance, the DMS IAM Role needs access to the secret and I am using this statement:

            {
                "actions": [                        
                            "secretsmanager:DescribeSecret",
                            "secretsmanager:GetSecretValue"
                            ],
                "resources": [
                            f"{self.secret.secret_full_arn}"
                        ]
            }

This statement fails because the secret_full_arn is not returning text after dash. For example:

The secret ARN is "arn:aws:secretsmanager:us-east-1:131578276461:secret:secret-for-data-platform-catalyst-Vg66GP". After the end of all CDK deployment, the IAM Role has secretsmanager permissions for "arn:aws:secretsmanager:us-east-1:131578276461:secret:secret-for-data-platform-catalyst". Note that "-Vg66GP" is missing.

I think this issue might be related to #11727

Image Image

Regression Issue

  • Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

I need the IAM role to have permissions for the real secret_arn

Current Behavior

When I use the statement below, secret_full_arn is not returning text after dash

            {
                "actions": [                        
                            "secretsmanager:DescribeSecret",
                            "secretsmanager:GetSecretValue"
                            ],
                "resources": [
                            f"{self.secret.secret_full_arn}"
                        ]
            }

Reproduction Steps

My secret is created here:

secret = secretsmanager.Secret(
    scope=stack, 
    id="RdsDatabaseSecret",
    description="Secret used for data catalyst modules",
    secret_name=config.secret_name,
    removal_policy=config.removal_policy,
    generate_secret_string=secretsmanager.SecretStringGenerator(
        secret_string_template='{"username":"rds_cdk_user"}',  
        generate_string_key="password",
        exclude_punctuation=True,
        exclude_characters='!@#$%^&*()[]{};:,.<>?/~`\\|'
    )
)

I use this secret as a class attribute:

source_endpoint_iam_role_stack = IamRoleStack(
                    scope=stack, 
                    construct_id="DmsSourceEnpointIamRole",
                    account_id=config.account_id, 
                    deployment_environment=config.deployment_environment, 
                    aws_region=config.aws_region,
                    aws_stack=f"dms.{config.aws_region}",
                    policy_name="cdk-iam-policy-dms-source-endpoint",
                    role_name="cdk-iam-role-dms-source-endpoint",
                    secret=secret
                    )

My class has this method to create iam statements:

  class IamRoleStack(cdk.Stack):
      """
      A stack that creates an AWS IAM Role with the necessary permissions.
      """
      def __init__(self, 
                   scope: Construct, 
                   construct_id: str,
                   account_id: str, 
                   deployment_environment: str, 
                   aws_region: str,
                   aws_stack: str,
                   effect: iam.Effect = iam.Effect.ALLOW,
                   policy_name: Optional[str] = None,
                   role_name: Optional[str] = None,
                   s3_bucket: s3.Bucket = None,
                   secret: secretsmanager.Secret = None,
                   **kwargs 
                   ) -> None:
          """
          Initializes the IAMRoleStack.
          
          :param scope: Scope in which this construct is defined.
          :param construct_id: Identifier for this stack.
          :param account_id: AWS account ID.
          :param deployment_environment: Deployment environment (e.g., "dev", "prod").
          :param aws_region: AWS region for resources.
          :param aws_stack: Stack name for naming convention.
          :param iam_permission_statements: A list of dictionaries containing "actions" and "resources".
          :param effect: Default effect for policy statements (default: ALLOW).
          :param policy_name: Default name used for IAM Policy (default: None).
          :param role_name: Default name used for IAM Role (default: None).
          :param s3_bucket: s3.Bucket resource (default: None).
          :param secret: secretsmanager.Secret resource (default: None).
          """
  
          super().__init__(scope, construct_id, **kwargs)
  
          # Instantiate class variables
          
          self.account_id = account_id
          self.deployment_environment = deployment_environment
          self.aws_region = aws_region
          self.aws_stack = aws_stack
          self.effect = effect
          self.policy_name = policy_name
          self.role_name = role_name
          self.s3_bucket = s3_bucket
          self.secret = secret
          self.generic_suffix = create_name(self.account_id, self.deployment_environment, self.aws_region)
  
      def create_iam_permission_statements(self) -> List[Dict]:
          """
          Creates a List with permissions (actions, resources) used in create_iam_policy_statements() function.
  
          :return iam_permissions (List[Dict]): A list of IAM permissions.
          """
  
          if self.s3_bucket:
              return [
                  {
                      "actions": [                        
                                  "s3:GetObject",
                                  "s3:GetObjectAcl",
                                  "s3:GetObjectVersion",
                                  "s3:GetObjectVersionAcl",
                                  "s3:PutObject",
                                  "s3:ListBucket",
                                  "s3:DeleteObject"
                                  ],
                      "resources": [
                                  self.s3_bucket.bucket_arn,
                                  f"{self.s3_bucket.bucket_arn}/*"
                              ]
                  }
                  ]
          
          elif self.secret:
              return [
                  {
                      "actions": [                        
                                  "secretsmanager:DescribeSecret",
                                  "secretsmanager:GetSecretValue"
                                  ],
                      "resources": [
                                  f"{self.secret.secret_full_arn}",
                                  f"{self.secret.secret_full_arn}-??????"
                              ]
                  }
                  ]

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.141.0

Framework Version

No response

Node.js Version

v22.1.0

OS

macOS 15.0 24A335

Language

Python

Language Version

3.13.0

Other information

No response

@RaulPontelloCaylent RaulPontelloCaylent added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Feb 18, 2025
@github-actions github-actions bot added the @aws-cdk/aws-secretsmanager Related to AWS Secrets Manager label Feb 18, 2025
@pahud pahud self-assigned this Feb 18, 2025
@pahud
Copy link
Contributor

pahud commented Feb 18, 2025

When you create the secret can you check if its full_secret_arn is correct?

e.g.

secret = secretsmanager.Secret(
    scope=stack, 
    id="RdsDatabaseSecret",
    description="Secret used for data catalyst modules",
    secret_name=config.secret_name,
    removal_policy=config.removal_policy,
    generate_secret_string=secretsmanager.SecretStringGenerator(
        secret_string_template='{"username":"rds_cdk_user"}',  
        generate_string_key="password",
        exclude_punctuation=True,
        exclude_characters='!@#$%^&*()[]{};:,.<>?/~`\\|'
    )
)

// CfnOutput the secret.secret_full_arn to verify

If it's correct then you can track when you receive this from a class, are you still able to get the full arn in the class?

@pahud pahud added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. effort/medium Medium work item – several days of effort p3 labels Feb 18, 2025
@pahud pahud removed their assignment Feb 18, 2025
@pahud pahud removed the needs-triage This issue or PR still needs to be triaged. label Feb 18, 2025
@RaulPontelloCaylent
Copy link
Author

RaulPontelloCaylent commented Feb 18, 2025

@pahud Both secret_arn and secret_full_arn give the same result and have the real Secret ARN.

I also tried this outputs outside the class:

    CfnOutput(stack, "SecretBefore", value=secret.secret_full_arn)

    source_endpoint_iam_role_stack = IamRoleStack(
                        scope=stack, 
                        construct_id="DmsSourceEnpointIamRole",
                        account_id=config.account_id, 
                        deployment_environment=config.deployment_environment, 
                        aws_region=config.aws_region,
                        aws_stack=f"dms.{config.aws_region}",
                        policy_name="cdk-iam-policy-dms-source-endpoint",
                        role_name="cdk-iam-role-dms-source-endpoint",
                        secret=secret
                        )
    
    CfnOutput(stack, "SecretAfter", value=source_endpoint_iam_role_stack.secret_full_arn)

Both CfnOutput above return the real Secret ARN but the DMS IAM Role still got the wrong Secret ARN.

Then, I tested this output inside the class:

  class IamRoleStack(cdk.Stack):
      """
      A stack that creates an AWS IAM Role with the necessary permissions.
      """
      def __init__(self, 
                   scope: Construct, 
                   construct_id: str,
                   account_id: str, 
                   deployment_environment: str, 
                   aws_region: str,
                   aws_stack: str,
                   effect: iam.Effect = iam.Effect.ALLOW,
                   policy_name: Optional[str] = None,
                   role_name: Optional[str] = None,
                   s3_bucket: s3.Bucket = None,
                   secret: secretsmanager.Secret = None,
                   **kwargs 
                   ) -> None:
          """
          Initializes the IAMRoleStack.
          
          :param scope: Scope in which this construct is defined.
          :param construct_id: Identifier for this stack.
          :param account_id: AWS account ID.
          :param deployment_environment: Deployment environment (e.g., "dev", "prod").
          :param aws_region: AWS region for resources.
          :param aws_stack: Stack name for naming convention.
          :param iam_permission_statements: A list of dictionaries containing "actions" and "resources".
          :param effect: Default effect for policy statements (default: ALLOW).
          :param policy_name: Default name used for IAM Policy (default: None).
          :param role_name: Default name used for IAM Role (default: None).
          :param s3_bucket: s3.Bucket resource (default: None).
          :param secret: secretsmanager.Secret resource (default: None).
          """
  
          super().__init__(scope, construct_id, **kwargs)
  
          # Instantiate class variables
          
          self.account_id = account_id
          self.deployment_environment = deployment_environment
          self.aws_region = aws_region
          self.aws_stack = aws_stack
          self.effect = effect
          self.policy_name = policy_name
          self.role_name = role_name
          self.s3_bucket = s3_bucket
          self.secret = secret
          self.generic_suffix = create_name(self.account_id, self.deployment_environment, self.aws_region)
  
          CfnOutput(scope, "SecretInside", value=self.secret.secret_full_arn)

This output inside the class gave me the wrong Secret ARN

@pahud
Copy link
Contributor

pahud commented Feb 18, 2025

Interesting.

class IamRoleStack(cdk.Stack):

OK so this looks like cross-stack reference.

Questions

  1. are you creating two stacks?
  2. Do you see the producer stack having an export which is imported by the consumer stack? What's in that export? I assume it's the secretName in the export?

Off the top of my head, if you are doing cross-stack reference. CDK essentially won't export the "concret" secret object and is very likely just export the name of the secret. When you import that in your consumer stack you are very likely only get the name of the secret so you won't have the fullarn which contains the last 6 chars.

Let me know if it is the case.

@RaulPontelloCaylent
Copy link
Author

@pahud I am not sure if I completely understood your last question.

Yes, I are creating two stacks, one for Secret and another for the DMS Endpoint

The DMS Endpoint stack will use the Secret stack as input

Secret stack is created like this:

    # Import libraries
    
    import aws_cdk as cdk
    from modules import config
    from modules.cdk_dpc_aws_iam_role.stack import IamRoleStack
    from modules.cdk_dpc_aws_dms_instance.stack import DmsInstanceStack
    from modules.cdk_dpc_aws_dms_endpoints.stack import DmsEndpointStack
    from modules.cdk_dpc_aws_dms_task.stack import DmsTaskStack
    from modules.data import *
    
    from aws_cdk import (
        aws_ec2 as ec2,
        aws_s3 as s3,
        aws_rds as rds,
        aws_secretsmanager as secretsmanager,
        aws_ec2 as ec2,
        CfnOutput
    )
    
  
    # Creating a new CDK application and environment configuration
    
    app = cdk.App()
    cdk_env = cdk.Environment(account=config.account_id, region=config.aws_region)
    stack = cdk.Stack(app, "DmsStack", env=cdk_env)

    # Create a Secret in Secrets Manager for RDS authentication
    
    secret = secretsmanager.Secret(
        scope=stack, 
        id="RdsDatabaseSecret",
        description="Secret used for data catalyst modules",
        secret_name=config.secret_name,
        removal_policy=config.removal_policy,
        generate_secret_string=secretsmanager.SecretStringGenerator(
            secret_string_template='{"username":"rds_cdk_user"}',  
            generate_string_key="password",
            exclude_punctuation=True,
            exclude_characters='!@#$%^&*()[]{};:,.<>?/~`\\|'
        )
    )

And I will use this secretsmanager.Secret object inside the IAM class like this:

    source_endpoint_iam_role_stack = IamRoleStack(
                        scope=stack, 
                        construct_id="DmsSourceEnpointIamRole",
                        account_id=config.account_id, 
                        deployment_environment=config.deployment_environment, 
                        aws_region=config.aws_region,
                        aws_stack=f"dms.{config.aws_region}",
                        policy_name="cdk-iam-policy-dms-source-endpoint",
                        role_name="cdk-iam-role-dms-source-endpoint",
                        secret=secret
                        )

@pahud
Copy link
Contributor

pahud commented Feb 18, 2025

Let's consider this sample in python

class ProducerStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs):
        super().__init__(scope, id, **kwargs)
        
        self.secret = secretsmanager.Secret(self, 'Secret',
            secret_name='my-secret',
            secret_string_value=SecretValue.unsafe_plain_text('supersecret')
        )
        
        CfnOutput(self, 'SecretOutput',
            value=self.secret.secret_full_arn
        )



class ConsumerStack(Stack):
    def __init__(self, scope: Construct, id: str, secret: secretsmanager.Secret, **kwargs):
        super().__init__(scope, id, **kwargs)

        CfnOutput(self, 'SecretOutput',
            value=secret.secret_full_arn
        )

and app.py

app = cdk.App()

producer = ProducerStack(app, "ProducerStack")
consumer = ConsumerStack(app, "ConsumerStack", secret=producer.secret)

on cdk deploy --all you should see

ProducerStack.SecretOutput and ConsumerStack.SecretOutput from both stacks like

arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:my-secret-8ASRal

And you should see this as well

ProducerStack.ExportsOutputRefSecretA720EF052D953DED = arn:aws:secretsmanager:us-east-1:ACCOUNT_ID:secret:my-secret-8ASRal

This indicates the fullArn is exported as a string from the producer stack that could be imported by the consumer stack.

Can you check if this works for you?

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Feb 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-secretsmanager Related to AWS Secrets Manager bug This issue is a bug. effort/medium Medium work item – several days of effort p3
Projects
None yet
Development

No branches or pull requests

2 participants