Merge branch 'master' into feature/how-it-works-cloudtrail-update

.circleci/config.yml

@@ -6,7 +6,7 @@ jobs:
   #
   build-and-deploy:
     machine:
-      image: ubuntu-1604:202007-01
+      image: ubuntu-2204:2022.04.1
       docker_layer_caching: false
 
     steps:

docs/first-steps/leverage-project-setup.md

@@ -273,7 +273,7 @@ You will end up with something that looks like this:
     |       └── 📂 <b>security-base</b>
     |           ├── 📄 account.tf
     |           ├── 📄 config.tf
-    |           ├── 📄 iam_access_analizer.tf
+    |           ├── 📄 iam_access_analyzer.tf
     |           ├── 📄 locals.tf
     │           └── 📄 variables.tf
     └── 📂 <b>shared</b>

docs/first-steps/post-deployment.md

@@ -46,7 +46,7 @@ As Natasha also has an IAM user for the `security` account besides the one in `m
 To be able to use the generated programmatic keys, you need to configure them in your local environment. To do that, run:
 
 ``` bash
-leverage credentials configure --type MANAGEMENT # or `SECURITY` depending on the credentials to configured
+leverage credentials configure --fetch-mfa-device --type MANAGEMENT # or `SECURITY` depending on the credentials to configured
 ```
 <pre><code><span class="fsg-timestamp">[12:28:12.111]</span> INFO     Loading configuration file.
 <span class="fsg-timestamp">[12:28:12.132]</span> INFO     Loading project environment configuration file.

docs/how-it-works/features/costs/costs.md

@@ -1,4 +1,4 @@
-# Cost Estmation & Optimization
+# Cost Estimation & Optimization
 
 ## Opportunity to optimize resources
 

docs/user-guide/features/sso/sso.md

@@ -5,12 +5,12 @@ Before deploying your AWS SSO definition in the project, it will first have to b
 !!! note ":books: [Prerequisites](https://docs.aws.amazon.com/singlesignon/latest/userguide/prereqs.html)"
 !!! info ":books: [Enable AWS SSO](https://docs.aws.amazon.com/singlesignon/latest/userguide/step1.html)"
 
-After that, choosing and configuring an Identity Provider (IdP) is the next step. For this we will make use of JumpCloud, as described in the [how it works](../../../how-it-works/features/sso/sso.md) section. These resources point to all requirements and procedures to have your JumpCloud account setup and synched with AWS SSO:
+After that, choosing and configuring an Identity Provider (IdP) is the next step. For this, we will make use of JumpCloud, as described in the [how it works](../../../how-it-works/features/sso/sso.md) section. These resources point to all requirements and procedures to have your JumpCloud account setup and synched with AWS SSO:
 
 !!! info ":books: [AWS JumpCloud support guide](https://docs.aws.amazon.com/singlesignon/latest/userguide/jumpcloud-idp.html)"
 !!! info ":books: [JumpCloud guide on how to configure as IdP for AWS SSO](https://docs.aws.amazon.com/singlesignon/latest/userguide/jumpcloud-idp.html)"
 
-Once this is set up the SSO layer can be safely deployed.
+Once this is set up, the SSO layer can be safely deployed.
 
 ## Preparing the project to use AWS SSO
 
@@ -81,6 +81,8 @@ This step needs to be performed every time the user works in a new account that
 
 ### 2. Logging in
 
+This step is executed as part of the previous one. So if the user has just configured SSO, this step is not required.
+
 Having SSO configured, the user will proceed to log in.
 
 This is achieved by running `leverage aws sso login`.
@@ -89,8 +91,6 @@ In this step, the user is prompted to manually authorize the log in process via
 
 When logging in, Leverage obtains a token from SSO. This token is later used to obtain the credentials needed for the layer the user is working on. This token has a relatively short life span to strike a balance between security and convenience for the user.
 
-This step is executed as part of the previous one. So if the user has just configured SSO, this step is not required.
-
 ### 3. Working on a layer
 
 When SSO is enabled in the project, Leverage will automatically figure out the required credentials for the current layer, and attempt to get them from AWS every time the user executes a command on it.

docs/user-guide/leverage-cli/reference/credentials.md

@@ -21,6 +21,7 @@ Backups of the previous configured credentials files are always created when ove
 ### Options
 * `--type`: Type of the credentials to set. Can be any of `BOOTSTRAP`, `MANAGEMENT` or `SECURITY`. This option is case insensitive. This option is required.
 * `--credentials-file`: Path to a `.csv` credentials file, as produced by the AWS Console, containing the user's programmatic access keys. If not given, the user will be prompted for the credentials.
+* `--fetch-mfa-device`: Retrieve an MFA device serial from AWS for the current user.
 * `--overwrite-existing-credentials`: If the type of credentials being configured is already configured, overwrite current configuration. Mutually exclusive option with `--skip-access-keys-setup`.
 * `--skip-access-keys-setup`: Skip the access keys configuration step. Continue on to setting up the accounts profiles. Mutually exclusive option with `--overwrite-existing-credentials`.
 * `--skip-assumable-roles-setup`: Don't configure each account profile to assume their specific role.

docs/user-guide/leverage-cli/reference/terraform.md

@@ -2,7 +2,7 @@
 
 The `terraform` command is a wrapper for a containerized installation of Terraform. It provides the Terraform executable with specific configuration values required by Leverage.
 
-It transparently handles authentication whether it is Multi-Factor or via Single Sign-On on behalf of the user in the commands that require it. SSO Authentication takes precedence over MFA when both are active. 
+It transparently handles authentication, whether it is Multi-Factor or via Single Sign-On, on behalf of the user in the commands that require it. SSO Authentication takes precedence over MFA when both are active. 
 
 Some commands can only be run at **layer** level and will not run anywhere else in the project.
 
@@ -13,7 +13,7 @@ The command can also be invoked via its shortened version `tf`.
 
 ### Usage
 ``` bash
-leverage terraform init [arguments]
+leverage terraform init [option] [arguments]
 ```
 
 Equivalent to `terraform init`.
@@ -22,6 +22,12 @@ All arguments given are passed as received to Terraform.
 
 Can only be run at **layer** level.
 
+[Layout validation](#validate-layout) is performed before actually initializing Terraform unless explicitly indicated against via the `--skip-validation` flag. 
+
+### Options
+* `--skip-validation`: Skips layout validation.
+
+
 ---
 ## `plan`
 
@@ -119,15 +125,14 @@ _Note:_ When `--sso` flag is used, the `--mfa` flag status is ignored.
 
 ### Usage
 ``` bash
-leverage terraform format [option]
+leverage terraform format [arguments]
 ```
 
 Equivalent to `terraform fmt -recursive`.
 
 Recursively format all files in the architecture to the Terraform code style.
 
-### Options
-* `--check`: Only perform format checking, do not overwrite the files.
+All arguments given are passed as received to Terraform.
 
 ---
 ## `validate`
@@ -141,6 +146,25 @@ Equivalent to `terraform validate`.
 
 Checks the infrastructure definition's consistency. 
 
+---
+## `validate-layout`
+
+### Usage
+``` bash
+leverage terraform validate-layout
+```
+
+Check the Terraform backend configuration in the code definition.
+
+Values checked:
+
+* Environment name in account configuration
+* S3 bucket key
+* AWS CLI profile name prefix
+* S3 Bucket name prefix
+* DynamoDB locking table name prefix
+
+
 ---
 ## `import`
 
@@ -154,3 +178,12 @@ Equivalent to `terraform import`.
 Import the resource with the given ID into the Terraform state at the given ADDRESS. 
 
 Can only be run at **layer** level.
+
+!!! info "zsh globbing"
+    Zsh users may need to prepend `noglob` to the import command for it to be recognized correctly, as an alternative, square brackets can be escaped as `\[\]`
+    **Examples:**
+    - Opt-1:  `leverage tf import module.s3_bucket.aws_s3_bucket.this\[0\] s3-bag-data-bucket` 
+    - Opt-2:  `noglob leverage tf import module.s3_bucket.aws_s3_bucket.this[0] s3-bag-data-bucket`  
+
+
+

docs/user-guide/leverage-cli/shell.md

@@ -1,30 +1,54 @@
 # Shell environment
 
-When launching a Terraform shell, Leverage provides the user with a completely isolated environment tailored to operate in the current project **layer** via a Docker container.
+When launching a Terraform shell, Leverage provides the user with a completely isolated environment tailored to operate in the current project via a Docker container.
+
+The whole project is mounted on a directory named after the value for `project_long` in the global configuration file, or simply named `"project"` if this value is not defined. A project named `myexample`, would be mounted in `/myexample`.
+
+The user's `~/.ssh` directory and `.gitconfig` file are also mounted on `/root/.ssh` and `/etc/gitconfig` respectively for convenience. Also, the credentials files (`credentials` and `config`) found in the project AWS credentials directory (`~/.aws/myexample`), are mapped to the locations given by the environment variables `AWS_SHARED_CREDENTIALS_FILE` and `AWS_CONFIG_FILE` respectively within the container. 
 
 ## Authentication
 Determining which credentials are needed to operate on a layer, and retrieving those credentials, may prove cumbersome for many complex layer definitions. In addition to that, correctly configuring them can also become a tedious an error prone process. For that reason Leverage automates this process upon launching the shell if requested by the user via the [`shell` command options](./reference/terraform.md#shell).
 
+Bear in mind, that an authenticated shell session's credentials are obtained for the layer in which the session was launched. These credentials may not be valid for other layers in which different roles need to be assumed or require more permissions.
+
 ### Multi-Factor authentication
+
+``` bash
+leverage terraform shell --mfa
+```
+
 If MFA authentication is required, Leverage will prompt the user for the required tokens for the layer or use the cached credentials if still valid.
 
 The user's programmatic keys must be configured beforehand via `leverage credentials configure` command.
 
 ### Single-Sign On
-If authentication via SSO is required, the user will need to [configure](./reference/aws.md#configure-sso) or [login](./reference/aws.md#sso-login) into SSO before launching the shell.
+
+If authentication via SSO is required, the user will need to [configure](./reference/aws.md#configure-sso) or [login](./reference/aws.md#sso-login) into SSO before launching the shell via
+
+``` bash
+leverage terraform shell --sso
+```
+
 
 ## Operations on the project's layer
-In order to operate in a project's layer, Terraform commands such as `plan` or `apply` will need to receive extra parameters providing the location of the files that contain the definition of the variables required by the layer. These files are:
+In order to operate in a project's layer, Terraform commands such as `plan` or `apply` will need to receive extra parameters providing the location of the files that contain the definition of the variables required by the layer. Usually, these files are:
 
 * the project global configuration file `common.tfvars`
 * the account configuration file `account.tfvars`
 * the terraform backend configuration file `backend.tfvars`
 
+In this case these parameters should take the form:
+``` bash
+-var-file=/myexample/config/common.tfvars -var-file=/myexample/account/config/account.tfvars -var-file=/myexample/account/config/backend.tfvars`
+```
 
-These parameters take the shape `-var-file=/common-config/common.tfvars -var-file=/config/account.tfvars -var-file=/config/backend.tfvars`
-
+Relative paths can prove useful when providing these locations. A layer definition may require more than just these files.
 
-So, for example, to apply changes in a layer, the complete command would be:
+So, for example, to apply changes on a standard Leverage Reference Architecture layer, the complete command would be:
+``` bash
+terraform apply -var-file=../../../config/common.tfvars -var-file=../../config/account.tfvars -var-file=../../config/backend.tfvars
+```
+However, when initializing Terraform different parameters are needed, so it should be run as:
 ``` bash
-terraform apply -var-file=/common-config/common.tfvars -var-file=/config/account.tfvars -var-file=/config/backend.tfvars
+terraform init -backend-config=../../config/backend.tfvars
 ```

docs/user-guide/ref-architecture-aws/configs.md

@@ -2,14 +2,14 @@
 
 !!! tips "Config files can be found under each `config` folders"
     - :file_folder: **Global config file** 
-    [`/config/common.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/config/common.config.example) 
+    [`/config/common.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/config/common.tfvars.example) 
     contains global context TF variables that we inject to TF commands which are used by all sub-directories such as 
-    `make plan` or `make apply` and which cannot be stored in `backend.config` due to TF.
+    `leverage terraform plan` or `leverage terraform apply` and which cannot be stored in `backend.tfvars` due to TF.
     - :file_folder: **Account config files** 
-        - [`backend.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/shared/config/backend.config)
+        - [`backend.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/shared/config/backend.tfvars)
          contains TF variables that are mainly used to configure TF backend but since
          `profile` and `region` are defined there, we also use them to inject those values into other TF commands.
-        - [`account.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/shared/config/account.config)
+        - [`account.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/shared/config/account.tfvars)
          contains TF variables that are specific to an AWS account.
 
 ## AWS Profile

docs/user-guide/ref-architecture-aws/tf-state-setup.md

@@ -11,14 +11,14 @@ you can get to work with anything else.
 
 !!! tips "Config files can be found under each `config` folders"
     - :file_folder: **Global config file** 
-    [`/config/common.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/config/common.config.example) 
+    [`/config/common.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/config/common.tfvars.example) 
     contains global context TF variables that we inject to TF commands which are used by all sub-directories such as 
-    `make plan` or `make apply` and which cannot be stored in `backend.config` due to TF.
+    `leverage terraform plan` or `leverage terraform apply` and which cannot be stored in `backend.config` due to TF.
     - :file_folder: **Account config files** 
-        - [`backend.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/shared/config/backend.config)
+        - [`backend.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/shared/config/backend.tfvars)
          contains TF variables that are mainly used to configure TF backend but since
          `profile` and `region` are defined there, we also use them to inject those values into other TF commands.
-        - [`account.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/shared/config/account.config)
+        - [`account.tfvars`](https://github.com/binbashar/le-tf-infra-aws/blob/master/shared/config/account.tfvars)
          contains TF variables that are specific to an AWS account.
 
 ## AWS Profile

docs/user-guide/ref-architecture-vault/configs.md

@@ -2,14 +2,14 @@
 
 !!! tips "Config files can be found under each `config` folders"
     - :file_folder: **Global config file** 
-    [`/config/common.tfvars`](https://github.com/binbashar/le-tf-vault/blob/master/config/common.config.example) 
+    [`/config/common.tfvars`](https://github.com/binbashar/le-tf-vault/blob/master/config/common.tfvars.example) 
     contains global context TF variables that we inject to TF commands which are used by all sub-directories such as 
-    `make plan` or `make apply` and which cannot be stored in `backend.config` due to TF.
+    `leverage terraform plan` or `leverage terraform apply` and which cannot be stored in `backend.tfvars` due to TF.
     - :file_folder: **Account config files** 
-        - [`backend.tfvars`](https://github.com/binbashar/le-tf-vault/blob/master/hcp/config/backend.config)
+        - [`backend.tfvars`](https://github.com/binbashar/le-tf-vault/blob/master/hcp/config/backend.tfvars)
          contains TF variables that are mainly used to configure TF backend but since
          `profile` and `region` are defined there, we also use them to inject those values into other TF commands.
-        - [`account.tfvars`](https://github.com/binbashar/le-tf-vault/blob/master/hcp/config/account.config)
+        - [`account.tfvars`](https://github.com/binbashar/le-tf-vault/blob/master/hcp/config/account.tfvars)
          contains TF variables that are specific to an AWS account.
 
 ## AWS Profile