Resolve Infinite Retry Loop on Database Outage and Enhance Buffer Shutdown Behavior

[janbjorge]

Addresses Issue #224, where the TimedOverflowBuffer would enter an infinite retry loop during database outages, blocking system shutdown and restart workflows. Key changes include:

1. Retry Backoff on Flush:

   • Implemented retry_backoff and shutdown_backoff mechanisms to manage retries during normal operations and shutdown.
   • Prevented indefinite retries by capping delays and integrating shutdown-aware logic.

2. Graceful Shutdown Enhancements:

   • Ensured pending flush operations exit cleanly during shutdown, even with unresolvable errors like database disconnections.
   • Integrated fine-grained control over shutdown behavior to balance recovery attempts with timely exits.

3. Test Coverage:

   • Added test_job_buffer_callback_exception_during_teardown to validate handling of exceptions during buffer shutdown.
   • Simulated database outages and verified buffer behavior with live retries, recovery, and controlled exits.

4. Documentation and Cleanup:

   • Updated docstrings for better clarity on buffer behavior and shutdown processes.
   • Improved parameter naming and code readability for backoff strategies.

Thanks to @AntonKarabaza for assisting with testing.