Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create lambda function to execute database loading #27

Closed
torimcd opened this issue Nov 1, 2023 · 1 comment · Fixed by #34
Closed

Create lambda function to execute database loading #27

torimcd opened this issue Nov 1, 2023 · 1 comment · Fixed by #34
Assignees
Labels
enhancement New feature or request

Comments

@torimcd
Copy link
Collaborator

torimcd commented Nov 1, 2023

In order to run the database load script in AWS environment, it should be wrapped in a lambda instead of a command line tool.

@frankinspace
Copy link
Member

@torimcd With the changes from #26 and #31 the entire hydrocron package (which includes the load data script) is now being built into a lambda container image. This should help take care a lot of the deployment concerns so that this ticket should become much easier to implement.

What needs to happen is:

  1. Create an entrypoint for lambda. Most examples use lambda_handler as the function name but it can be whatever. This function takes the lambda event as input (JSON). The function needs to parse the lambda event to extract the parameters (instead of relying on CLI argparse). All the other code you have should be ok.
  2. Define a new lambda function in terraform. You can use the timeseries lambda as an example, but make sure to use the dynamo-write-policy I prepared instead of the dynamo-read-policy (which means you'll also probably want to define a new execution role specifically for the load data lambda too). You'll also have to update the command part to point to the new lambda handler function.
  3. Update the documentation for operations so they know how to invoke the lambda. It's probably best to just assume access to the AWS console and invoking the lambda that way. If you want to get fancy you could come up with an example using a AWS CLI command to execute the lambda

@torimcd torimcd self-assigned this Nov 9, 2023
@frankinspace frankinspace linked a pull request Nov 10, 2023 that will close this issue
4 tasks
@github-project-automation github-project-automation bot moved this from 🔖 Ready to ✅ Done in SOTO PI 23.4 Nov 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants