Feature/issue 78 VPC Endpoint for Dynamo

[nikki-t]

Github Issue: #78

Description

Configure the timeseries, load_data and load_data-granule Lambda functions for VPC access. This will ensure that Lambda network traffic is isolated to a single VPC and traverse the DynamoDB VPC endpoint created by NGAP.

Overview of work done

Configured all three Lambda functions for VPC access.

This required an update to the AWS provider (~> 4.0) and the Terraform CLI (1.7.3).

Overview of verification done

Existing functionality is not impacted and current unit tests pass.

Overview of integration done

Configuration was deployed to the SIT environment and confirmed existing functionality.

Reach GeoJSON request:

{"status":"200 OK","time":239.655,"hits":3,"results":{"csv":"","geojson":{"type":"FeatureCollection","features":[{"properties":{"reach_id":"73282400171","time_str":"2023-07-28T22:10:37Z","wse":"211.178","slope":"-0.0007516914999999999"},"geometry":{},"type":"Feature"},{"properties":{"reach_id":"73282400171","time_str":"2023-09-08T15:40:49Z","wse":"82.082","slope":"-1.0184e-06"},"geometry":{},"type":"Feature"},{"properties":{"reach_id":"73282400171","time_str":"2023-09-29T12:25:52Z","wse":"206.49349999999998","slope":"-0.0007833485000000001"},"geometry":{},"type":"Feature"}]}}}

Set up flow logs for the timeseries Lambda function elastic network interface and confirmed data traversing the interface and sending/receiving data from an IP address in the CIDR range for the DynamoDB Endpoint.

Example log:

2024-03-14T10:47:48.000-04:00 2 <ACCOUNT_ID> eni-xxxxxxxx <DYNAMODB> <LAMBDA> 443 12133 6 217 253126 1710427668 1710427669 ACCEPT OK
2024-03-14T10:47:48.000-04:00 2 <ACCOUNT_ID> eni-xxxxxxxx <LAMBDA> <DYNAMODB> 12133 443 6 153 64613 1710427668 1710427669 ACCEPT OK

Ran Reachability Analyzer with the Lambda function's elastic network interface as the source and the DynamoDB VPC Endpoint as the destination and confirmed traffic can get from the Lambda function to the Endpoint.

PR checklist:

• Linted
• Updated unit tests
• Updated changelog
• Integration testing

See Pull Request Review Checklist for pointers on reviewing this pull request

[nikki-t]

I was able to confirm VPC configuration for the Lambda and view traffic traversing the elastic network interface associated with the Lambda. I can see the Lambda's elastic network interface IP address querying an IP address in one of the CIDR ranges associated with the DynamoDB VPC endpoint and I can see the return response from the endpoint to the Lambda function. I am waiting on final confirmation from AWS Support and will set the PR as ready for review.

[nikki-t]

Confirmed configuration with AWS support and ran a reachability analysis to see if the Lambda function elastic network interface can reach the DynamoDB VPC Endpoint.

@torimcd - do you want to run a test in the SIT environment for the database operations to make sure everything still functions as it should? I have the feature/issue-78 branch currently deployed to that environment.

[frankinspace]

Do we actually need this? I just removed it in one of the other branches and things seemed to work fine.

Suggested change

cryptography = "42.0.0"

[torimcd]

Looks good to me, tested the db operations in SIT and everything works as it should.

[nikki-t]

Thanks @torimcd for testing the db operations in SIT. I think this is ready to be merged unless we want to wait and add the VPC configuration to the cnm_handler Lambda?

[torimcd]

@nikki-t the cnm_handler lambda only invokes the load_granule lambda, it doesn't interact with dynamodb directly. Not sure how that effects whether or not to include in the vpc?

[nikki-t]

@torimcd - That makes sense. I wasn't sure if we wanted all of our Lambdas configured for the VPC or maybe it's preferable to only configure the ones with DynamoDB access for the VPC? @frankinspace - Do you have a preference?

[frankinspace]

If it's not accessing dynamo no need to be in VPC