[Exploration] Testing Serverless applications in the cloud

[jfuss]

While writing unit tests and end-to-end tests for Serverless applications is similar to traditional applications, other types of tests (such as integration tests, acceptance tests, component tests) are more challenging.
We want to explore ways this can be made easier, and need your feedback. Here are a few ideas we are thinking about -

Manage timeouts and retries for tests targeting event driven applications
Optimize test execution time
Auto generate test payloads for a set of AWS services
Invoke using auto generated payloads
Asserting on event payloads
Injecting failures
Set up a test environment to run tests in an isolated manner

Please feel free to drop your ideas/requests pertaining to integration testing in the comments. Specifically, what are the sticky issues you run into when writing tests for Serverless applications on AWS? Do the ideas above accurately address the pain points?

This was migrated from Issues. Original Issue can be found at #3631

[igorlg]

+1 to Auto generate test payloads for a set of AWS services! Anything that can be done to simplify this step is most welcome!

One way to do this could be to scan the template and suggest event pauloads based on the Event key of each function resource!