Allow customizable backoff factor and backoff max

[psaia]

Initial problem

We'd like to run tests with variable run-durations more frequently. This can be almost accomplished by setting a more rapid cron (e.g. 30s) along side a much longer windowtimeout.

However, when using a larger windowtimeout w/ the previously hard-coded backoff factor, it can create a significant delay before the HC status is actually updated.

Changes

• Allow for the max and min retry timeout to be customizable (to make it non exponential)
• Allow for the retry factor to be customizable
• Add clearer log indicating a workflow for a HC cannot be found
• Misc. code linting

[psaia]

@tekenstam (Sorry, pinging you due to recent activity) - Is there anything further I can do to have this PR reviewed?

[codecov]

Codecov Report

Attention: Patch coverage is 76.92308% with 6 lines in your changes missing coverage. Please review.

Project coverage is 67.99%. Comparing base (2ff4c95) to head (809e21f).
Report is 10 commits behind head on master.

Files	Patch %	Lines
internal/controllers/healthcheck_controller.go	76.92%	3 Missing and 3 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #215      +/-   ##
==========================================
+ Coverage   65.64%   67.99%   +2.35%     
==========================================
  Files           3        3              
  Lines        1141      978     -163     
==========================================
- Hits          749      665      -84     
+ Misses        313      232      -81     
- Partials       79       81       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.