Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Potential flakes: 20180810 #11

Closed
joyeecheung opened this issue Aug 10, 2018 · 5 comments
Closed

Potential flakes: 20180810 #11

joyeecheung opened this issue Aug 10, 2018 · 5 comments

Comments

@joyeecheung
Copy link
Member

joyeecheung commented Aug 10, 2018 •
edited by BridgeAR
Loading

Produced with ncu-ci walk pr --stats, counting tests that failed more than 1 PR

Potential flaky tests:

-------------------------------- Stats --------------------------------
Reason     parallel/test-module-loading-globalpaths
Type       JS_TEST_FAILURE
Failed PR  5 (#21857, #22209, #22020, #22152, #22212)
Appeared   test-packetnet-ubuntu1604-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16260/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16304/
--------------------------------------------------------------------------------
Reason     parallel/test-tls-server-verify
Type       JS_TEST_FAILURE
Failed PR  5 (#22122, #21857, #22074, #22184, #21975)
Appeared   test-softlayer-alpine36_container-x64-1, test-digitalocean-alpine37_container-x64-2, test-softlayer-centos6-x64-2, test-softlayer-centos7-x64-1, test-digitalocean-debian8-x64-1, test-softlayer-debian8-x86-1, test-softlayer-debian9-x64-1, test-digitalocean-fedora26-x64-1, test-rackspace-fedora27-x64-1, test-digitalocean-ubuntu1404-x86-1, test-softlayer-ubuntu1404-x64-1, test-digitalocean-ubuntu1604-x86-2, test-rackspace-ubuntu1604-x64-1, test-joyent-ubuntu1710-x64-2, test-digitalocean-ubuntu1804_container-x64-2, test-digitalocean-freebsd11-x64-1, test-macstadium-macos10.11-x64-2, test-osuosl-ubuntu1404-ppc64_le-1, test-joyent-smartos16-x64-1, test-joyent-smartos17-x64-2, test-softlayer-ubuntu1604_sharedlibs_container-x64-5, test-digitalocean-ubuntu1604_sharedlibs_container-x64-6, test-joyent-ubuntu1604_sharedlibs_container-x64-3, test-digitalocean-ubuntu1604_sharedlibs_container-x64-10, test-digitalocean-ubuntu1604_sharedlibs_container-x64-1, test-linuxonecc-rhel72-s390x-3, test-osuosl-aix61-ppc64_be-2, test-azure_msft-win10-x64-4, test-rackspace-win2008r2-x64-5, test-azure_msft-win2016-x64-5, test-rackspace-win2012r2-x64-4, test-requireio_mhdawson-debian9-armv6l_pi1p-1, test-requireio_jasnell-debian9-armv7l_pi2-1, test-requireio_pivotalagency-debian9-arm64_pi3-2, test-rackspace-debian8-x64-1, test-digitalocean-debian9-x64-1, test-rackspace-fedora26-x64-1, test-digitalocean-fedora27-x64-1, test-softlayer-ubuntu1404-x86-1, test-digitalocean-ubuntu1404-x64-1, test-joyent-ubuntu1710-x64-1, test-softlayer-ubuntu1804_container-x64-1, test-macstadium-macos10.11-x64-1, test-osuosl-ubuntu1404-ppc64_le-2, test-joyent-smartos16-x64-2, test-joyent-smartos17-x64-1, test-joyent-ubuntu1604_sharedlibs_container-x64-5, test-osuosl-aix61-ppc64_be-1, test-packetnet-ubuntu1604-x64-2, test-azure_msft-win10-x64-1, test-azure_msft-win2016-x64-3, test-requireio_bengl-debian9-armv6l_pi1p-1, test-requireio_notthetup_sayanee-debian9-arm64_pi3-1, test-softlayer-centos6-x64-1, test-rackspace-centos7-x64-1, test-digitalocean-ubuntu1604_sharedlibs_container-x64-7, test-softlayer-ubuntu1604_sharedlibs_container-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16259/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16266/
--------------------------------------------------------------------------------
Reason     parallel/test-trace-event-promises
Type       JS_TEST_FAILURE
Failed PR  4 (#22104, #22122, #22146, #22145)
Appeared   test-packetnet-centos7-arm64-2, test-scaleway-ubuntu1604-armv7l-2, test-packetnet-ubuntu1604-arm64-1, test-packetnet-centos7-arm64-1, test-scaleway-ubuntu1604-armv7l-1, test-packetnet-ubuntu1604-arm64-2, test-digitalocean-freebsd11-x64-1, test-digitalocean-alpine36_container-x64-1, test-digitalocean-alpine37_container-x64-2, test-softlayer-centos6-x64-2, test-softlayer-centos7-x64-1, test-rackspace-debian8-x64-1, test-softlayer-debian8-x86-1, test-digitalocean-debian9-x64-1, test-rackspace-fedora26-x64-1, test-rackspace-fedora27-x64-1, test-softlayer-ubuntu1404-x86-1, test-softlayer-ubuntu1404-x64-1, test-digitalocean-ubuntu1604-x86-1, test-rackspace-ubuntu1604-x64-1, test-joyent-ubuntu1710-x64-1, test-softlayer-ubuntu1804_container-x64-1, test-macstadium-macos10.11-x64-1, test-osuosl-ubuntu1404-ppc64_le-2, test-joyent-smartos16-x64-2, test-joyent-smartos17-x64-1, test-digitalocean-ubuntu1604_sharedlibs_container-x64-8, test-digitalocean-ubuntu1604_sharedlibs_container-x64-6, test-joyent-ubuntu1604_sharedlibs_container-x64-5, test-digitalocean-ubuntu1604_sharedlibs_container-x64-10, test-digitalocean-ubuntu1604_sharedlibs_container-x64-1, test-scaleway-ubuntu1604-armv7l-3, test-linuxonecc-rhel72-s390x-3, test-osuosl-aix61-ppc64_be-1, test-azure_msft-win10-x64-1, test-rackspace-win2008r2-x64-6, test-azure_msft-win2016-x64-3, test-rackspace-win2012r2-x64-7, test-requireio_bengl-debian9-armv6l_pi1p-2, test-requireio_svincent-debian9-armv7l_pi2-2, test-requireio_williamkapke-debian9-arm64_pi3-3, test-osuosl-aix61-ppc64_be-2, test-joyent-smartos16-x64-1, test-joyent-smartos17-x64-2, test-softlayer-ubuntu1604_sharedlibs_container-x64-2, test-softlayer-ubuntu1604_sharedlibs_container-x64-5, test-joyent-ubuntu1604_sharedlibs_container-x64-3, test-digitalocean-debian8-x64-1, test-softlayer-debian9-x64-1, test-digitalocean-fedora26-x64-1, test-digitalocean-ubuntu1404-x86-1, test-digitalocean-ubuntu1604-x86-2, test-digitalocean-ubuntu1804_container-x64-2, test-osuosl-ubuntu1404-ppc64_le-1, test-macstadium-macos10.11-x64-2, test-azure_msft-win10-x64-5, test-rackspace-win2008r2-x64-3, test-azure_msft-win2016-x64-2, test-rackspace-win2012r2-x64-5
First CI   https://ci.nodejs.org/job/node-test-pull-request/16275/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16279/
--------------------------------------------------------------------------------
Reason     parallel/test-trace-events-fs-sync
Type       JS_TEST_FAILURE
Failed PR  4 (#22175, #22189, #22152, #22185)
Appeared   test-linuxonecc-rhel72-s390x-3, test-azure_msft-win10-x64-5
First CI   https://ci.nodejs.org/job/node-test-pull-request/16257/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16311/
--------------------------------------------------------------------------------
Reason     parallel/test-worker-dns-terminate
Type       JS_TEST_FAILURE
Failed PR  3 (#21875, #22189, #22198)
Appeared   test-osuosl-aix61-ppc64_be-2, test-digitalocean-ubuntu1604_sharedlibs_container-x64-10
First CI   https://ci.nodejs.org/job/node-test-pull-request/16268/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16285/
--------------------------------------------------------------------------------
Reason     sequential/test-benchmark-http
Type       JS_TEST_FAILURE
Failed PR  3 (#22145, #22209, #22106)
Appeared   test-digitalocean-fedora27-x64-1, test-packetnet-ubuntu1604-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16279/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16305/
--------------------------------------------------------------------------------
Reason     sequential/test-net-bytes-per-incoming-chunk-overhead
Type       JS_TEST_FAILURE
Failed PR  3 (#22104, #22074, #21875)
Appeared   test-digitalocean-ubuntu1404-x64-1
First CI   https://ci.nodejs.org/job/node-test-pull-request/16256/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16268/
--------------------------------------------------------------------------------
Reason     async-hooks/test-callback-error
Type       JS_TEST_FAILURE
Failed PR  2 (#22184, #22106)
Appeared   test-digitalocean-fedora27-x64-1
First CI   https://ci.nodejs.org/job/node-test-pull-request/16265/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16305/
--------------------------------------------------------------------------------
Reason     async-hooks/test-statwatcher
Type       JS_TEST_FAILURE
Failed PR  2 (#22122, #21975)
Appeared   test-digitalocean-ubuntu1604_sharedlibs_container-x64-10, test-azure_msft-win2016-x64-4, test-joyent-ubuntu1710-x64-1
First CI   https://ci.nodejs.org/job/node-test-pull-request/16277/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16326/
--------------------------------------------------------------------------------
Reason     parallel/test-bootstrap-modules
Type       JS_TEST_FAILURE
Failed PR  2 (#22128, #21857)
Appeared   test-packetnet-ubuntu1604-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16236/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16260/
--------------------------------------------------------------------------------
Reason     parallel/test-fs-readfilesync-pipe-large
Type       JS_TEST_FAILURE
Failed PR  2 (#22020, #22152)
Appeared   test-packetnet-ubuntu1604-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16297/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16331/
--------------------------------------------------------------------------------
Reason     parallel/test-fs-syncwritestream
Type       JS_TEST_FAILURE
Failed PR  2 (#21857, #22020)
Appeared   test-packetnet-ubuntu1604-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16260/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16297/
--------------------------------------------------------------------------------
Reason     parallel/test-http-chunk-problem
Type       JS_TEST_FAILURE
Failed PR  2 (#22074, #22152)
Appeared   test-packetnet-ubuntu1604-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16261/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16303/
--------------------------------------------------------------------------------
Reason     parallel/test-http2-pipe
Type       JS_TEST_FAILURE
Failed PR  2 (#22128, #22020)
Appeared   test-rackspace-win2008r2-x64-4, test-rackspace-win2008r2-x64-3
First CI   https://ci.nodejs.org/job/node-test-pull-request/16253/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16297/
--------------------------------------------------------------------------------
Reason     parallel/test-stdio-pipe-redirect
Type       JS_TEST_FAILURE
Failed PR  2 (#22177, #22202)
Appeared   test-azure_msft-win10-x64-3, test-rackspace-win2008r2-x64-1, test-azure_msft-win2012r2-x64-2, test-azure_msft-win2012r2-x64-1, test-azure_msft-win2016-x64-2, test-rackspace-win2012r2-x64-9, test-azure_msft-win10-x64-5, test-azure_msft-win2016-x64-5
First CI   https://ci.nodejs.org/job/node-test-pull-request/16258/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16327/
--------------------------------------------------------------------------------
Reason     sequential/test-benchmark-net
Type       JS_TEST_FAILURE
Failed PR  2 (#22145, #22209)
Appeared   test-packetnet-ubuntu1604-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16279/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16296/
--------------------------------------------------------------------------------
Reason     sequential/test-benchmark-tls
Type       JS_TEST_FAILURE
Failed PR  2 (#21975, #22152)
Appeared   test-packetnet-ubuntu1604-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16254/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16303/

Build failures:

Reason     ERROR: Error fetching remote repo 'origin'
Type       JENKINS_FAILURE
Failed PR  2 (#22115, #22222)
Appeared   test-softlayer-centos6-x64-1, test-rackspace-centos7-x64-1, test-softlayer-ubuntu1404-x64-1, test-joyent-ubuntu1710-x64-2, test-softlayer-centos6-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16240/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16321/
--------------------------------------------------------------------------------
Reason     java.io.IOException: Backing channel JNLP4-connect connection from ... is disconnected.
Type       JENKINS_FAILURE
Failed PR  2 (#22104, #22222)
Appeared   test-packetnet-ubuntu1604-x64-2, test-rackspace-win2008r2-x64-5
First CI   https://ci.nodejs.org/job/node-test-pull-request/16275/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16321/

Linter failures (likely not a flaky, if not a build flake):

Reason     - /usr/home/iojs/build/workspace/node-test-linter/lib/dgram.js
Type       JS_TEST_FAILURE
Failed PR  4 (#22128, #21975, #21875, #22104)
Appeared   test-joyent-freebsd10-x64-2
First CI   https://ci.nodejs.org/job/node-test-pull-request/16253/
Last CI    https://ci.nodejs.org/job/node-test-pull-request/16256/
@BridgeAR
Copy link
Member

I just updated the entries above to split them into three sections.

  1. Flaky tests
  2. Build related failures
  3. Linter failures

@BridgeAR
Copy link
Member

@nodejs/trace-events PTAL at: parallel/test-trace-event-promises and parallel/test-trace-events-fs-sync

@nodejs/http @nodejs/http2 PTAL at: parallel/test-http-chunk-problem, parallel/test-http2-pipe and sequential/test-benchmark-net

@nodejs/fs PTAL at: parallel/test-fs-readfilesync-pipe-large and parallel/test-fs-syncwritestream

@nodejs/workers PTAL at: parallel/test-worker-dns-terminate

@nodejs/modules PTAL at: parallel/test-module-loading-globalpaths

@nodejs/build-infra PTAL at the jenkins issues.

@lundibundi
Copy link
Member

I think parallel/test-worker-dns-terminate needs a stress test. It should be fixed based on nodejs/node#21327 (comment) and this PR nodejs/node#22189 no longer fails after rebase and this one nodejs/node#22198 fails but doesn't have a fix in it.
Local --repeat 1920 on linux box have also passed just fine.

@joyeecheung
Copy link
Member Author

@BridgeAR By the way I think if the failure is caused by the --worker job (most test-packetnet-ubuntu1604-x64-2 failures there), it probably needs help from @nodejs/workers instead - and most of them are actually ENOENT under --worker so it could have something to do with how tmpdir resolves there.

@joyeecheung
Copy link
Member Author

Closing because..there is no reason to keep it open?
By the way the --worker test failures reduced significantly these two days. I'll post the new results grabbed today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@joyeecheung @BridgeAR @lundibundi