Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky test on self-hosted runner #418

Closed
kwinkunks opened this issue Feb 28, 2023 · 0 comments · Fixed by #419
Closed

Fix flaky test on self-hosted runner #418

kwinkunks opened this issue Feb 28, 2023 · 0 comments · Fixed by #419
Assignees
Labels
bug Something isn't working

Comments

@kwinkunks
Copy link
Member

On Azure, the test test_response_correlation.test_axes_labels() is flaky, timing out half the time with the default 10 s limit. Here is one example:

=================================== FAILURES ===================================
_______________________________ test_axes_labels _______________________________

mock_data = None
dash_duo = <dash.testing.composite.DashComposite object at 0x2b4017208970>

    def test_axes_labels(mock_data, dash_duo):
        """test_axis_labels loads two different plots and checks that axes are
        labelled correctly"""
    
        plugin = setup_plugin(
            dash_duo, __name__, ResponseCorrelation, window_size=(1024, 2048)
        )
    
        # find the right ensemble which has mock data prepared for this test
        wanted_ensemble_name = "default3"
        select_ensemble(dash_duo, plugin, wanted_ensemble_name)
    
        wanted_responses = ["WOPR:OP1", "FOPR"]
    
        # we only see one response at a time, so we choose and check one after the
        # other
        for wanted_response in wanted_responses:
            response_selector_id = plugin.uuid("parameter-selector-multi-resp")
            select_by_name(dash_duo, f"#{response_selector_id}", wanted_response)
    
            plot_id = plugin.uuid("response-overview")
    
            # check that y axis label spells out "Value"
>           dash_duo.wait_for_text_to_equal(f"#{plot_id} text.ytitle", "Value")

tests/views/test_response_correlation.py:34: 
.
.
.
E       selenium.common.exceptions.TimeoutException: Message: text -> Value not found within 10s

../test-kenv/root/lib64/python3.8/site-packages/selenium/webdriver/support/wait.py:87: TimeoutException

Following tests depend on the same plot. Perhaps it takes about 10 s to render (seems long but 🤷 ) so propose increasing timeout limit for this one wait.

@kwinkunks kwinkunks self-assigned this Feb 28, 2023
@kwinkunks kwinkunks added the bug Something isn't working label Feb 28, 2023
@kwinkunks kwinkunks added this to SCOUT Feb 28, 2023
@kwinkunks kwinkunks moved this to In Progress in SCOUT Feb 28, 2023
@berland berland moved this from In Progress to In Review in SCOUT Mar 1, 2023
@github-project-automation github-project-automation bot moved this from In Review to Done in SCOUT Mar 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant