feat: adding Progress Bar to Benchmark script

[AAfghahi]

SUMMARY

This adds a progress bar to the current benchmark script.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Untitled.mov

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

• Has associated issue:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

[codecov]

Codecov Report

Merging #15719 (4726481) into master (fe68e45) will decrease coverage by 0.22%.
The diff coverage is 50.00%.

❗ Current head 4726481 differs from pull request most recent head 20c9c01. Consider uploading reports for the commit 20c9c01 to get more accurate results

@@            Coverage Diff             @@
##           master   #15719      +/-   ##
==========================================
- Coverage   76.91%   76.69%   -0.23%     
==========================================
  Files         983      983              
  Lines       51609    51607       -2     
  Branches     6988     6988              
==========================================
- Hits        39696    39580     -116     
- Misses      11690    11804     +114     
  Partials      223      223              

Flag	Coverage Δ
hive	?
mysql	81.55% <50.00%> (+<0.01%)	⬆️
postgres	81.57% <50.00%> (+<0.01%)	⬆️
presto	?
python	81.66% <50.00%> (-0.43%)	⬇️
sqlite	81.18% <50.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
superset/utils/mock_data.py	25.18% <50.00%> (+0.36%)	⬆️
superset/db_engines/hive.py	0.00% <0.00%> (-82.15%)	⬇️
superset/db_engine_specs/hive.py	69.44% <0.00%> (-17.07%)	⬇️
superset/db_engine_specs/presto.py	83.36% <0.00%> (-6.53%)	⬇️
superset/views/database/mixins.py	81.03% <0.00%> (-1.73%)	⬇️
superset/connectors/sqla/models.py	88.26% <0.00%> (-1.65%)	⬇️
superset/db_engine_specs/base.py	88.28% <0.00%> (-0.40%)	⬇️
superset/models/core.py	89.79% <0.00%> (-0.27%)	⬇️
superset/utils/core.py	88.12% <0.00%> (-0.13%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe68e45...20c9c01. Read the comment docs.

[betodealmeida]

Keep this empty line.

[betodealmeida]

These changes were merged already, right?

[betodealmeida]

So, there's a few subtle bugs in your implementation. The critical one is that for every model you define entities as an empty list. But you only add the entities to the session after you've processed all models (line 198). This means you'll only add the entities of the last model in the for model in models loop.

I'd also only show the progress bar if you have something to add.

Something like this:

while min_entities <= limit:
    downgrade(revision=down_revision)
    print(f"Running with at least {min_entities} entities of each model")
    for model in models:
        missing = min_entities - model_rows[model]
        if missing > 0:
            entities: List[Model] = []
            bar = ChargingBar(f"Adding {missing} entities to the {model.__name__} model", max=missing)
            try:
                for entity in add_sample_rows(session, model, missing)
                    entities.append(entity)
                    bar.next()
            except Exception:
                session.rollback()
                raise
            bar.finish()
            model_rows[model] = min_entities

            # add all created entities
            session.add_all(entities)
            session.commit()

            if auto_cleanup:
                new_models[model].extend(entities)

[betodealmeida]

How does the print in 183 interact with the bar? I think it might be better to reverse the order of 182,183.

Can you post a screencast of the progress bar in action? :)

[AAfghahi]

I just switched and pushed up, it does look a lot better. Putting a screencast in the main portion.

[betodealmeida]

So cool!