[FEA] Start to do some scale testing

[revans2]

Is your feature request related to a problem? Please describe.
Our current scale testing is really around running NDS at larger and larger scale factors to see what fails. This is fine but it does not exercise many different aspects of scale. Like how many aggregations can we do in a single operation before things start to fall over.

This is an epic to come up with a set of scripts that will let us start to test the RAPIDs not just for larger amount of data, but also for more and more complex operations. The first part of creating a data gen script for integration tests is done, but there are is a lot more work to do to finish this.

• [FEA] Write entry point to generate data for scale testing. #8813
• [FEA] Create a BigDataChecking tool #8724
• [FEA] Write Initial Set of scale test queries + application #8814
• [FEA] Figure out how to monitor host memory usage in local mode for scale testing + add launch scripts #8815
• [FEA] Write the rest of the scale test queries #8816
• [FEA] Figure out how to monitor scale tests on standlone cluster #8817
• [FEA] Figure out how to monitor scale tests on YARN #8818
• I will leave kubernetes scale testing support for a later time.