From d783d6716bd040caa02994dec1a0bade85b0b29f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Juan=20Mat=C3=ADas=20Kungfoo=20de=20la=20C=C3=A1mara=20Beo?= =?UTF-8?q?vide?= Date: Fri, 14 Apr 2023 15:58:50 -0300 Subject: [PATCH] ISSUE-158 | add new aws account (#172) * BASH-158 Improving proc doc and moving it to its own page * BASH-158 Conflicts resolved * ISSUE-158 Fixed typo * ISSUE-158 Improved expression * ISSUE-158 Improved expression * ISSUE-158 Improved expression --- docs/first-steps/add-aws-accounts.md | 466 +++++++++++++++++++++++++++ docs/first-steps/post-deployment.md | 306 ------------------ mkdocs.yml | 1 + 3 files changed, 467 insertions(+), 306 deletions(-) create mode 100644 docs/first-steps/add-aws-accounts.md diff --git a/docs/first-steps/add-aws-accounts.md b/docs/first-steps/add-aws-accounts.md new file mode 100644 index 00000000..da926bfc --- /dev/null +++ b/docs/first-steps/add-aws-accounts.md @@ -0,0 +1,466 @@ +# Add AWS Accounts + +If necessary you can easily add new AWS accounts to your Leverage project by following these steps. + +For this example `apps-prd` will be created in region `us-east-1`. + +## Create the account in your Organization + +1. Go to `management/global/organizations`. + +2. Edit the `locals.tf` file to add the account to the local `accounts` variable. + ```yaml + accounts = { + + [...] + + apps-prd = { + email = "aws+apps-prd@yourcompany.com", + parent_ou = "apps" + } + } + ``` + + Note `apps` organizational unit (OU) is being used as parent OU. If a new OU has to be used here, it has to be created by adding it to `organizational_units` structure in the same file. + +3. Run the [Terraform workflow](https://leverage.binbash.com.ar/user-guide/ref-architecture-aws/workflow/) to apply the new changes. + + Basically: + + ```shell + leverage terraform apply + ``` + +4. Add the new account to the `config/common.tfvars` file. + + The new account ID should be got from the previous step. + + Usi it to update the file: + + ```shell + accounts = { + + [...] + + apps-prd = { + email = "aws+apps-prd@yourcompany.com", + id = "" + } + } + ``` + +5. If SSO is being used in this project. + + Permissions for SSO access have to be granted before we can step forward. + + Add the right permissions in file `management/global/sso/account_assignments.tf`. + + For the example: + + ```yaml + { + account = var.accounts.apps-prd.id, + permission_set_arn = module.permission_sets.permission_sets["Administrator"].arn, + permission_set_name = "Administrator", + principal_type = "GROUP", + principal_name = "AWS_Administrators" + }, + + { + account = var.accounts.apps-prd.id, + permission_set_arn = module.permission_sets.permission_sets["DevOps"].arn, + permission_set_name = "DevOps", + principal_type = "GROUP", + principal_name = "AWS_DevOps" + }, + + { + account = var.accounts.apps-prd.id, + permission_set_arn = module.permission_sets.permission_sets["Developer_FullAccess"].arn, + permission_set_name = "Developer_FullAccess", + principal_type = "GROUP", + principal_name = "AWS_Developers" + }, + + ``` + + Note your needs can vary, these permissions are just an example, please be careful with what you are granting here. + + Apply changes: + ```shell + leverage terraform apply + ``` + + Now you need to get the new permissions locally: + + ```shell + leverage aws configure sso + ``` + +Now, you have to create the initial directory structure for this new account, *as explained below*. + +## Setup and apply the layers for the new account + + For this example, we will create the `apps-prd` account structure by using the `apps-devstg` as a template: + +1. Ensure you are at the root of this repository + +2. Create the initial directory structure for the new account: + + ```shell + mkdir -p apps-prd/{global,us-east-1} + ``` + +3. Set up the config files: + + 1. Create the config files for this account: + + ```shell + cp -r apps-devstg/config apps-prd/config + ``` + + 2. Open `apps-prd/config/backend.tfvars` and replace any occurrences of `devstg` with `prd`. + + (basically, `apps-devstg` is being replaced with the new name `apps-prd`) + + + 3. Do the same with `apps-prd/config/account.tfvars` + + 4. If **no SSO** is implemented in the project (i.e. OAAR is being used): + + 1. Open up `apps-prd/config/backend.tfvars` again and replace this: + ```yaml + profile = "bb-apps-prd-devops" + ``` + with this: + ```yaml + profile = "bb-apps-prd-oaar" + ``` + + 2. In the step above, we are switching to the OAAR (OrganizationalAccountAccessRole) role because we are working with a brand new account that is empty, so, the only way to access it programmatically is through the OAAR role. + + 3. Now it's time to configure your OAAR credentials (if haven't already done so). For that you can follow the steps in [this section](https://leverage.binbash.com.ar/first-steps/management-account/#update-the-bootstrap-credentials) of the official documentation. + +4. Create the `base-tf-backend` layer: + + 1. Copy the layer from an existing one: + + From the repository root run: + ```shell + cp -r apps-devstg/us-east-1/base-tf-backend apps-prd/us-east-1/base-tf-backend + ``` + + !!! info + If the source layer was already initialized you should delete the previous Terraform setup using `sudo rm -rf .terraform*` in the target layer's directory. + + 2. Go to the `apps-prd/us-east-1/base-tf-backend` directory, open the `config.tf` file and comment the S3 backend block: + + E.g.: + ```yaml + #backend "s3" { + # key = "apps-devstg/tf-backend/terraform.tfstate" + #} + ``` + + 3. Now run the [Terraform workflow](https://leverage.binbash.com.ar/user-guide/ref-architecture-aws/workflow/) to initialize and + apply this layer. + + The flag `--skip-validation` is needed here since the bucket does not yet exist. + + ```shell + leverage terraform init --skip-validation + leverage terraform apply + ``` + + 4. Open the `config.tf` file again uncommenting the block commented before and replacing `devstg` with `prd`: + + E.g.: + ```yaml + backend "s3" { + key = "apps-prd/tf-backend/terraform.tfstate" + } + ``` + + 5. To finish with the backend layer, re-init to move the `tfstate` to the new location. + + Run: + ```shell + leverage terraform init + ``` + + Terraform will detect that you are trying to move from a local to a remote state and will ask for confirmation. + + ```shell + Initializing the backend... + Acquiring state lock. This may take a few moments... + Do you want to copy existing state to the new backend? + Pre-existing state was found while migrating the previous "local" backend to the + newly configured "s3" backend. No existing state was found in the newly + configured "s3" backend. Do you want to copy this state to the new "s3" + backend? Enter "yes" to copy and "no" to start with an empty state. + + Enter a value: + + ``` + + Enter `yes` and hit enter. + +5. Create the `base-identities` layer: + + 1. Copy the layer from an existing one: + + From the repository root run: + ```shel + cp -r apps-devstg/global/base-identities apps-prd/global/base-identities` + ``` + + !!! info + If the source layer was already initialized you should delete the previous Terraform setup using `sudo rm -rf .terraform*` in the target layer's directory. + + 2. Go to the `apps-prd/global/base-identities` directory and open the `config.tf` file. Replace any occurrences of `devstg` with `prd` + + E.g. this line should be: + ```yaml + backend "s3" { + key = "apps-prd/identities/terraform.tfstate" + } + ``` + + 3. Init the layer + + ```shell + leverage terraform init + ``` + + 4. Import the OAAR role + + Run this command: + ```shell + leverage terraform import module.iam_assumable_role_oaar.aws_iam_role.this OrganizationAccountAccessRole + ``` + + 5. Finally apply the layer + + ```shell + leverage terraform apply + ``` + +6. Create the `security-base` layer: + + 1. Copy the layer from an existing one: + + From the repository root run: + ```shell + cp -r apps-devstg/us-east-1/security-base apps-prd/us-east-1/security-base + ``` + + !!! info + If the source layer was already initialized you should delete the previous Terraform setup using `sudo rm -rf .terraform*` in the target layer's directory. + + 2. Go to the `apps-prd/us-east-1/security-base` directory and open the `config.tf` file replacing any occurrences of `devstg` with `prd` + + E.g. this line should be: + ```yaml + backend "s3" { + key = "apps-prd/security-base/terraform.tfstate" + } + ``` + + 3. Init and apply the layer + + ```shell + leverage tf init + leverage tf apply + ``` + +7. Create the `base-network` layer: + + 1. Copy the layer from an existing one: + + From the repository root run: + ```shell + cp -r apps-devstg/us-east-1/base-network apps-prd/us-east-1/base-network + ``` + + !!! info + If the source layer was already initialized you should delete the previous Terraform setup using `sudo rm -rf .terraform*` in the target layer's directory. + + 2. Go to the `apps-prd/us-east-1/base-network` directory and open the `config.tf` file replacing any occurrences of `devstg` with `prd` + + E.g. this line should be: + ```yaml + backend "s3" { + key = "apps-prd/network/terraform.tfstate" + } + ``` + + 3. Open the file `locals.tf` and set the new account's CIDRs + + E.g. + ```yaml + vpc_cidr_block = "172.19.0.0/20" + azs = [ + "${var.region}a", + "${var.region}b", + #"${var.region}c", + #"${var.region}d", + ] + + private_subnets_cidr = ["172.19.0.0/21"] + private_subnets = [ + "172.19.0.0/23", + "172.19.2.0/23", + #"172.19.4.0/23", + #"172.19.6.0/23", + ] + + public_subnets_cidr = ["172.19.8.0/21"] + public_subnets = [ + "172.19.8.0/23", + "172.19.10.0/23", + #"172.19.12.0/23", + #"172.19.14.0/23", + ] + ``` + + Note here only two azs are enabled, if needed uncomment the other ones in the three structures. + + 3. Init and apply the layer + + ```shell + leverage tf init + leverage tf apply + ``` + + 4. VPC Peering to Shared account + + Edit file `shared/us-east-1/base-network/config.tf` and add provider and remote state for the created account: + ```yaml + provider "aws" { + alias = "apps-prd" + region = var.region + profile = "${var.project}-apps-prd-devops" + shared_credentials_file = "~/.aws/${var.project}/config" + } + + data "terraform_remote_state" "apps-prd-vpcs" { + for_each = { + for k, v in local.apps-prd-vpcs : + k => v if !v["tgw"] + } + + backend = "s3" + + config = { + region = lookup(each.value, "region") + profile = lookup(each.value, "profile") + bucket = lookup(each.value, "bucket") + key = lookup(each.value, "key") + } + } + + ``` + + Edit file `shared/us-east-1/base-network/locals.tf` and under + ```yaml + # + # Data source definitions + # + ``` + + ...add the related structure: + ```yaml + apps-prd-vpcs = { + apps-prd-base = { + region = var.region + profile = "${var.project}-apps-prd-devops" + bucket = "${var.project}-apps-prd-terraform-backend" + key = "apps-prd/network/terraform.tfstate" + tgw = false + } + } + ``` + + Edit file `shared/us-east-1/base-network/vpc_peerings.tf` and add the peering definition: + ```yaml + # + # VPC Peering: AppsPrd VPC => Shared VPC + # + module "vpc_peering_apps_prd_to_shared" { + source = "github.com/binbashar/terraform-aws-vpc-peering.git?ref=v4.0.1" + + for_each = { + for k, v in local.apps-prd-vpcs : + k => v if !v["tgw"] + } + + providers = { + aws.this = aws + aws.peer = aws.apps-prd + } + + this_vpc_id = module.vpc.vpc_id + peer_vpc_id = data.terraform_remote_state.apps-prd-vpcs[each.key].outputs.vpc_id + + this_rts_ids = concat(module.vpc.private_route_table_ids, module.vpc.public_route_table_ids) + peer_rts_ids = concat( + data.terraform_remote_state.apps-prd-vpcs[each.key].outputs.public_route_table_ids, + data.terraform_remote_state.apps-prd-vpcs[each.key].outputs.private_route_table_ids + ) + + auto_accept_peering = true + + tags = merge(local.tags, { + "Name" = "${each.key}-to-shared", + "PeeringRequester" = each.key, + "PeeringAccepter" = "shared" + }) + } + + ``` + + Apply the changes (be sure to CD into `shared/us-east-1/base-network` layer for doing this): + + ```shell + leverage terraform apply + ``` + +8. If no SSO is implemented in the project (i.e. OAAR is being used), switch back from OAAR to DevOps role + + 1. Open up `apps-prd/config/backend.tfvars` + + Replace this: + ```yaml + profile = "bb-apps-prd-oaar" + ``` + + with this: + + ```yaml + profile = "bb-apps-prd-devops" + ``` + + !!! info + This is needed because we only want to use the OAAR role for exceptional cases, not on a daily basis. + + 3. Now, let's configure your DevOps credentials (if you haven't already done so). + + 1. Log into your security account, create programmatic access keys, and enable MFA. + + 2. Then run: `leverage credentials configure --fetch-mfa-device --type SECURITY` + + 3. The command above should prompt for your programmatic keys and, with those, Leverage should be able to configure your AWS config and credentials files appropriately. + +9. That should be it. At this point you should have the following: + + 1. A brand-new AWS account + + 2. Configuration files that are needed for any layer that is created under this account + + 3. A Terraform State Backend for this new account + + 4. Roles and policies (base identities) that are necessary to access the new account + + 5. The base networking stuff + + diff --git a/docs/first-steps/post-deployment.md b/docs/first-steps/post-deployment.md index 1f7c2a9b..bb4cd581 100644 --- a/docs/first-steps/post-deployment.md +++ b/docs/first-steps/post-deployment.md @@ -98,312 +98,6 @@ That's it! You are ready to roll on your own now! :raised_hand: If a layer was already set with BOOTSTRAP credentials, when changing the credential type Terraform has to be reconfigured: `leverage tf init -reconfigure`. -## Add New AWS Accounts - -If necessary you can easily add new AWS accounts to your Leverage project by following these steps. - -For this example `apps-prd` will be created. - -#### Create the account in your Organization - -1. Go to `management/global/organizations`. - -2. Edit the `locals.tf` file to add the account to the local `accounts` variable. - ```yaml - accounts = { - - [...] - - apps-prd = { - email = "aws+apps-prd@yourcompany.com", - parent_ou = "apps" - } - } - ``` - - Note `apps` organizational unit (OU) is being used as parent OU. If a new OU has to be used here, it has to be created be adding it to `organizational_units` structure in the same file. - -3. Run the [Terraform workflow](https://leverage.binbash.com.ar/user-guide/ref-architecture-aws/workflow/) to apply the new changes. - - Basically: - - ```shell - leverage terraform apply - ``` - -4. Add the new account to the `config/common.tfvars` file. - - The new account ID should be got from the previous step. - - Using it update the file: - - ```shell - accounts = { - - [...] - - apps-prd = { - email = "aws+apps-prd@yourcompany.com", - id = "" - } - } - ``` - -5. If SSO is being used in this project. - - Permissions for SSO access have to be granted before we can step forward. - - Add the right permissions in file `management/global/sso/account_assignments.tf`. - - For the example: - - ```yaml - { - account = var.accounts.apps-prd.id, - permission_set_arn = module.permission_sets.permission_sets["Administrator"].arn, - permission_set_name = "Administrator", - principal_type = "GROUP", - principal_name = "AWS_Administrators" - }, - - { - account = var.accounts.apps-prd.id, - permission_set_arn = module.permission_sets.permission_sets["DevOps"].arn, - permission_set_name = "DevOps", - principal_type = "GROUP", - principal_name = "AWS_DevOps" - }, - - { - account = var.accounts.apps-prd.id, - permission_set_arn = module.permission_sets.permission_sets["Developer_FullAccess"].arn, - permission_set_name = "Developer_FullAccess", - principal_type = "GROUP", - principal_name = "AWS_Developers" - }, - - ``` - - Note that your needs can vary, and these permissions are just an example. Please be careful with what you are granting here. - - Apply changes: - ```shell - leverage terraform apply - ``` - - Now you need to get the new permissions locally: - - ```shell - leverage aws configure sso - ``` - -From here, to make sure it integrates correctly, you will most likely want to create the initial directory structure for this new account, *as explained below*. - -#### Create the initial directory structure for a new account - - For this example, we will set up the `apps-prd` account by using the `apps-devstg` as a template: - -1. Ensure you are at the root of this repository - -2. Create the initial directory structure for the new account: - - ``` - mkdir -p apps-prd/{global,us-east-1} - ``` - -3. Set up the config files: - - 1. Create the config files for this account: `cp -r apps-devstg/config apps-prd/config` - - 2. Open `apps-prd/config/backend.tfvars` and replace any occurrences of `devstg` with `prd`. - - (basically, `apps-devstg` is being replaced with the new name `apps-prd`) - - - 3. Do the same with `apps-prd/config/account.tfvars` - - 4. If **no SSO** is implemented in the project (i.e. OAAR is being used): - - 1. Open up `apps-prd/config/backend.tfvars` again and replace this: - ``` - profile = "bb-apps-prd-devops" - ``` - with this: - ``` - profile = "bb-apps-prd-oaar" - ``` - - 2. In the step above, we are switching to the OAAR (OrganizationalAccountAccessRole) role because we are working with a brand new account that is empty, so, the only way to access it programmatically is through the OAAR role. - - 3. Now it's time to configure your OAAR credentials (if haven't already done so). For that you can follow the steps in [this section](https://leverage.binbash.com.ar/first-steps/management-account/#update-the-bootstrap-credentials) of the official documentation. - -4. Create the `base-tf-backend` layer: - - 1. Copy the layer from an existing one: - - From the repository root run: - ```shell - cp -r apps-devstg/us-east-1/base-tf-backend apps-prd/us-east-1/base-tf-backend - ``` - - !!! info - If the source layer was already initialized you should delete the previous Terraform setup using `sudo rm -rf .terraform*` in the target layer's directory. - - 2. Go to the `apps-prd/us-east-1/base-tf-backend` directory and open the `config.tf` file and comment the the S3 backend block: - - E.g.: - ``` - #backend "s3" { - # key = "apps-devstg/tf-backend/terraform.tfstate" - #} - ``` - - 3. Now run the [Terraform workflow](https://leverage.binbash.com.ar/user-guide/ref-architecture-aws/workflow/) to initialize and apply this layer. - - The flag `--skip-validation` is needed here since the bucket does not yet exist. - - ```shell - leverage terraform init --skip-validation - leverage terraform apply - ``` - - 4. Open the `config.tf` file again uncommenting the block commented before and replacing `devstg` with `prd`: - - E.g.: - ``` - backend "s3" { - key = "apps-prd/tf-backend/terraform.tfstate" - } - ``` - - 5. To finish with the backend layer, re-init to move the `tfstate` to the new location. - - Run: - ```shell - leverage terraform init - ``` - - Terraform will detect that you are trying to move from a local to a remote state and will ask for confirmation. - - ```shell - Initializing the backend... - Acquiring state lock. This may take a few moments... - Do you want to copy existing state to the new backend? - Pre-existing state was found while migrating the previous "local" backend to the - newly configured "s3" backend. No existing state was found in the newly - configured "s3" backend. Do you want to copy this state to the new "s3" - backend? Enter "yes" to copy and "no" to start with an empty state. - - Enter a value: - - ``` - - Enter `yes` and hit enter. - -5. Create the `base-identities` layer: - - 1. Copy the layer from an existing one: - - From the repository root run: - ```shel - cp -r apps-devstg/global/base-identities apps-prd/global/base-identities` - ``` - - !!! info - If the source layer was already initialized you should delete the previous Terraform setup using `sudo rm -rf .terraform*` in the target layer's directory. - - 2. Go to the `apps-prd/global/base-identities` directory and open the `config.tf` file. Replace any occurrences of `devstg` with `prd` - - E.g. this line should be: - ```yaml - backend "s3" { - key = "apps-prd/identities/terraform.tfstate" - } - ``` - - 3. Init the layer - - ```shell - leverage terraform init - ``` - - 4. Import the OAAR role - - Run this command: - ```shell - leverage terraform import module.iam_assumable_role_oaar.aws_iam_role.this OrganizationAccountAccessRole - ``` - - 5. Finally apply the layer - - ```shell - leverage terraform apply - ``` - -6. Create the `security-base` layer: - - 1. Copy the layer from an existing one: - - From the repository root run: - ```shell - cp -r apps-devstg/us-east-1/security-base apps-prd/us-east-1/security-base - ``` - - !!! info - If the source layer was already initialized you should delete the previous Terraform setup using `sudo rm -rf .terraform*` in the target layer's directory. - - 2. Go to the `apps-prd/us-east-1/security-base` directory and open the `config.tf` file replacing any occurrences of `devstg` with `prd` - - E.g. this line should be: - ```yaml - backend "s3" { - key = "apps-prd/security-base/terraform.tfstate" - } - ``` - - 3. Init and apply the layer - - ```shell - leverage tf init - leverage tf apply - ``` - -7. If no SSO is implemented in the project (i.e. OAAR is being used), switch back from OAAR to DevOps role. - - 1. Open up `apps-prd/config/backend.tfvars` - - Replace this: - ```yaml - profile = "bb-apps-prd-oaar" - ``` - - with this: - - ```yaml - profile = "bb-apps-prd-devops" - ``` - - !!! info - This is needed because we only want to use the OAAR role for exceptional cases, not on a daily basis. - - 2. Now, let's configure your DevOps credentials (if you haven't already done so). - - 1. Log into your security account, create programmatic access keys, and enable MFA. - - 2. Then run: `leverage credentials configure --fetch-mfa-device --type SECURITY` - - 3. The command above should prompt for your programmatic keys and, with those, Leverage should be able to configure your AWS config and credentials files appropriately. - -8. That should be it. At this point you should have the following: - - 1. A brand-new AWS account. - - 2. Configuration files that are needed for any layer that is created under this account. - - 3. A Terraform State Backend for this new account. - - 4. Roles and policies (base identities) that are necessary to access the new account. - ## Next steps Now you not only have a fully functional landing zone configuration deployed, but also are able to interact with it using your own AWS SSO credentials. diff --git a/mkdocs.yml b/mkdocs.yml index bd1b1c72..9671b639 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -166,6 +166,7 @@ nav: - Orchestrate the Management account: "first-steps/management-account.md" - Orchestrate the Security and Shared accounts: "first-steps/security-and-shared-accounts.md" - Post-deployment: "first-steps/post-deployment.md" + - Add AWS Accounts: "first-steps/add-aws-accounts.md" - How it works: - Reference Architecture: