-
Notifications
You must be signed in to change notification settings - Fork 6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Docs] De-flake doctests #37162
[Docs] De-flake doctests #37162
Conversation
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@@ -67,6 +67,7 @@ async frameworks like aiohttp, aioredis, etc. | |||
# NOTE: The outputs from the previous code block can show up in subsequent tests. | |||
# To prevent flakiness, we wait for the async calls finish. | |||
import time | |||
print("Sleeping...") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is this for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Usually, there are outputs from the previous code block (which asynchronously prints out stuff), and this code block is intended to catch them. However, sometimes everything actually gets caught in the previous testoutput
block. In this case, there are no outputs from this code block, and the test errors.
This print ensures there are always some outputs from this code block.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
data changes lgtm
@@ -296,7 +296,7 @@ For more details, see the :ref:`Batch inference user guide <batch_inference_home | |||
Saving Datasets containing torch tensors | |||
---------------------------------------- | |||
|
|||
Datasets containing torch tensors can be saved to files, like parquet or numpy. | |||
Datasets containing torch tensors can be saved to files, like parquet or numpy. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Datasets containing torch tensors can be saved to files, like parquet or numpy. | |
You can save Datasets containing torch Tensors to files, like parquet or NumPy. |
Signed-off-by: Balaji Veeramani <balaji@anyscale.com> :book: Doctest (CPU) fails 25% of runs due to a few flaky tests. This PR deflakes those tests. Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
Signed-off-by: Balaji Veeramani <balaji@anyscale.com> :book: Doctest (CPU) fails 25% of runs due to a few flaky tests. This PR deflakes those tests. Signed-off-by: e428265 <arvind.chandramouli@lmco.com>
Why are these changes needed?
:book: Doctest (CPU)
fails 25% of runs due to a few flaky tests. This PR deflakes those tests.This test is flaky because the order of columns is non-deterministic.
This test is flaky because, in rare occasions, all outputs from the previous code block actually show up in the previous testoutput. (Usually, a few of the outputs show up in this testoutput).
These tests are flaky because the outputs contain extra logs if the dataset isn't cached.
Related issue number
Checks
git commit -s
) in this PR.scripts/format.sh
to lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/
under thecorresponding
.rst
file.