Retry upload 3 times in case of failure #703
Conversation
Codecov Report
@@ Coverage Diff @@
## main #703 +/- ##
==========================================
- Coverage 89.62% 89.59% -0.03%
==========================================
Files 166 166
Lines 13276 13278 +2
==========================================
- Hits 11898 11897 -1
- Misses 1378 1381 +3
... and 3 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
pyproject.toml
Outdated
| @@ -67,7 +67,7 @@ test = [ | |||
| "coverage[toml]>=7.0.5", | |||
| "pytest-cov>=4.0.0", | |||
| "scikit-learn>=1.0", | |||
| "tensorflow>=2.9.1", | |||
| "tensorflow>=2.9.1,<2.13.0", | |||
There was a problem hiding this comment.
so folks can never use a higher version of tensorflow?
There was a problem hiding this comment.
needs to be fixed sperately
There was a problem hiding this comment.
This req is in the test extras. But if they do happen to use tf >= 2.13.0 our method https://github.com/rungalileo/dataquality/blob/9b8af3636decb6bb5a7828df7fe6ac7e755188f3/dataquality/utils/keras.py#L46C1-L54C6 will fail and it seems to be used everywhere.
It's a prob in TF so I opened an issue on TF tensorflow/tensorflow#61204 (comment)
dataquality/clients/api.py
Outdated
| max_retries = 3 if retry else 1 | ||
| try: | ||
| for attempt in Retrying(stop=stop_after_attempt(max_retries)): | ||
| with attempt: | ||
| res = RequestType.get_method(request.value)( | ||
| url, | ||
| json=body, | ||
| params=params, | ||
| headers=header, | ||
| data=data, | ||
| timeout=timeout, | ||
| files=files, | ||
| ) | ||
| except RetryError as e: | ||
| raise GalileoException( | ||
| f"Failed to make request after {max_retries} attempts. " f"Error: {e}" | ||
| ) |
There was a problem hiding this comment.
let's chat about this. i would just use the tenacity decorator. 1 line instead of 17
Adds 3 tries for retrying to upload in case of failure.