a visual workflow builder used to orchestrate AWS services, automate processes, and build serverless applications. Create standardized processes around handling failures, retries, and parallelization to free developer teams for higher-value business logic. - https://aws.amazon.com/step-functions
AWS step functions allows you to create a State Machine, a series of steps or 'states', each perform some actions and retain information from step 'state' to step.
- Lambda functions are stateless this makes more complicated or multi part operations difficult to orchestrate in lambda
-
ETL tool - multiple ETL steps can be run linearly or in parallel why would you use step functions vs traditionally tools? decoupling compute form workflow
-
Data science and machine learning data science and machine learning projects often have multiple data updating and processing steps that must be run to update and publish a model example:
-
Collect data
-
prepare data:
-
handle nulls ie input
-
one hot encode categoricals
-
-
integer encode ordinal values
-
model train/publish
Etc Aws step function SDK for data science - https://github.com/aws/aws-step-functions-data-science-sdk-python
-
-
yelps use case - https://engineeringblog.yelp.com/2017/11/breaking-down-the-monolith-with-aws-step-functions.html
traditionally - Amazon states language
new this month - Workflow studio GUI
we will now create a demo step function that will call one of two lambda functions based on input and send a email reporting which function was run steps:
- outline state machine
- create lambda functions
- create AMI to allow state-machine to run lambda functions
- create state machine
- test state machine
- what are some other use cases for step functions
Most of the code was modified from examples Demo on the Be a better Dev Youtube channel