Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve reporting of harness errors in ShadowRealm tests #49387

Merged
merged 3 commits into from
Nov 27, 2024

Get stack traces for errors in most shadowrealm tests

21bc331
Select commit
Loading
Failed to load commit list.
Merged

Improve reporting of harness errors in ShadowRealm tests #49387

Get stack traces for errors in most shadowrealm tests
21bc331
Select commit
Loading
Failed to load commit list.
Community-TC Integration / tools/ unittests (Python 3.8) succeeded Nov 27, 2024 in 6m 56s

Community-TC (pull_request)

Unit tests for tools running under Python 3.8, excluding wptrunner

Details

View task in Taskcluster | View logs in Taskcluster | View task group in Taskcluster

Task Status

Started: 2024-11-27T19:31:58.718Z
Resolved: 2024-11-27T19:38:51.285Z
Task Execution Time: 6 minutes, 52 seconds, 567 milliseconds
Task Status: completed
Reason Resolved: completed
RunId: 0

Artifacts

- public/logs/live_backing.log
- public/logs/live.log


[taskcluster 2024-11-27 19:31:58.776Z] Task ID: UoPY1bcTSKmULoAYOzuGIA
[taskcluster 2024-11-27 19:31:58.776Z] Worker ID: 7149837768781909020
[taskcluster 2024-11-27 19:31:58.776Z] Worker Group: us-east4-c
[taskcluster 2024-11-27 19:31:58.776Z] Worker Node Type: projects/757942385826/machineTypes/n2-standard-4
[taskcluster 2024-11-27 19:31:58.776Z] Worker Pool: proj-wpt/ci
[taskcluster 2024-11-27 19:31:58.776Z] Worker Version: 46.1.0
[taskcluster 2024-11-27 19:31:58.776Z] Public IP: 34.48.126.180
[taskcluster 2024-11-27 19:31:58.776Z] Hostname: proj-wpt-ci-aoaqgc9xs8ed8nh50fh12a

[taskcluster 2024-11-27 19:31:59.563Z] === Task Starting ===
+ '[' '' ']'
+ '[' -d /etc/profile.d ']'
+ for i in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/01-locale-fix.sh ']'
+ . /etc/profile.d/01-locale-fix.sh
+++ /usr/bin/locale-check C.UTF-8
++ eval
+ for i in /etc/profile.d/*.sh
+ '[' -r /etc/profile.d/libvirt-uri.sh ']'
+ . /etc/profile.d/libvirt-uri.sh

...(231 lines hidden)...

wptserve/tests/test_config.py .......xx................................. [ 97%]

..                                                                       [ 97%]

wptserve/tests/test_replacement_tokenizer.py ..............              [ 99%]

wptserve/tests/test_request.py .....                                     [ 99%]

wptserve/tests/test_response.py ..                                       [ 99%]

wptserve/tests/test_stash.py ..                                          [100%]



=============================== warnings summary ===============================

ci/tc/tests/test_valid.py: 64 warnings

  /home/test/web-platform-tests/tools/.tox/py38/lib/python3.8/site-packages/jsonschema/validators.py:1115: DeprecationWarning: The metaschema specified by $schema was not found. Using the latest draft to validate, but this will raise an error in the future.

    cls = validator_for(schema)



-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html



---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                  Stmts   Miss Branch BrPart  Cover

---------------------------------------------------------------------------------------

__init__.py                                               1      0      0      0   100%

ci/__init__.py                                            0      0      0      0   100%

ci/check_for_updated_refs.py                             31      7     12      2    79%

ci/jobs.py                                               88     22     48      3    73%

ci/macos_color_profile.py                                19     19      8      0     0%

ci/make_hosts_file.py                                    12     12      0      0     0%

ci/manifest_build.py                                    148    148     42      0     0%

ci/regen_certs.py                                        37     37      4      0     0%

ci/run_tc.py                                            242    242     91      0     0%

ci/taskcluster-run.py                                    74     74     28      0     0%

ci/tc/__init__.py                                         0      0      0      0   100%

ci/tc/decision.py                                       250     62    121      5    77%

ci/tc/download.py                                        71     71     26      0     0%

ci/tc/github_checks_output.py                            13      5      2      1    60%

ci/tc/sink_task.py                                       39     39     14      0     0%

ci/tc/taskgraph.py                                      124      9     70      9    91%

ci/update_built.py                                       37     37     14      0     0%

conftest.py                                              12      0      2      0   100%

docker/__init__.py                                        0      0      0      0   100%

docker/frontend.py                                       88     88     36      0     0%

docker/retry.py                                          39     39     10      0     0%

gitignore/__init__.py                                     0      0      0      0   100%

gitignore/gitignore.py                                  188    188    120      0     0%

lint/__init__.py                                          1      0      0      0   100%

lint/fnmatch.py                                          21      2     10      2    81%

lint/lint.py                                            646     99    397     23    84%

lint/rules.py                                           299      5      0      0    98%

localpaths.py                                            36      1      2      1    95%

manifest/XMLParser.py                                    91      2     14      1    97%

manifest/__init__.py                                      1      0      0      0   100%

manifest/download.py                                    138    111     50      0    14%

manifest/item.py                                        234     17     42     11    89%

manifest/jsonlib.py                                      44     21     14      5    48%

manifest/log.py                                           6      1      0      0    83%

manifest/manifest.py                                    226     83    100     13    60%

manifest/mputil.py                                        7      1      2      1    78%

manifest/sourcefile.py                                  587     46    269     35    90%

manifest/spec.py                                         43     43      6      0     0%

manifest/testpaths.py                                    56     56     18      0     0%

manifest/typedata.py                                    194     38     86     13    79%

manifest/update.py                                       46     29      8      1    33%

manifest/utils.py                                        54     13     16      5    69%

manifest/vcs.py                                         199    156     66      1    17%

metadata/__init__.py                                      0      0      0      0   100%

metadata/meta/__init__.py                                 0      0      0      0   100%

metadata/meta/schema.py                                  13      0      4      0   100%

metadata/schema.py                                       52     52     22      0     0%

metadata/webfeatures/__init__.py                          0      0      0      0   100%

metadata/webfeatures/schema.py                           49     49     18      0     0%

metadata/yaml/__init__.py                                 0      0      0      0   100%

metadata/yaml/load.py                                     9      0      0      0   100%

serve/__init__.py                                         1      0      0      0   100%

serve/serve.py                                          657    246    196     35    59%

serve/test_functional.py                                 46      2      2      1    94%

serve/test_serve.py                                      67      4      0      0    94%

serve/wave.py                                            78     78     22      0     0%

wave/__init__.py                                          0      0      0      0   100%

wave/configuration_loader.py                             40     40      8      0     0%

wave/data/__init__.py                                     0      0      0      0   100%

wave/data/client.py                                       5      5      0      0     0%

wave/data/device.py                                       6      6      0      0     0%

wave/data/event_listener.py                               7      7      0      0     0%

wave/data/exceptions/__init__.py                          0      0      0      0   100%

wave/data/exceptions/duplicate_exception.py               2      2      0      0     0%

wave/data/exceptions/invalid_data_exception.py            2      2      0      0     0%

wave/data/exceptions/not_found_exception.py               2      2      0      0     0%

wave/data/exceptions/permission_denied_exception.py       2      2      0      0     0%

wave/data/http_polling_client.py                          8      8      0      0     0%

wave/data/http_polling_event_listener.py                  9      9      0      0     0%

wave/data/session.py                                     50     50     20      0     0%

wave/network/__init__.py                                  0      0      0      0   100%

wave/network/api/__init__.py                              0      0      0      0   100%

wave/network/api/api_handler.py                          74     74     22      0     0%

wave/network/api/devices_api_handler.py                 155    155     54      0     0%

wave/network/api/general_api_handler.py                  45     45     18      0     0%

wave/network/api/results_api_handler.py                 186    186     48      0     0%

wave/network/api/sessions_api_handler.py                328    328    128      0     0%

wave/network/api/tests_api_handler.py                   191    191     68      0     0%

wave/network/http_handler.py                             85     85     28      0     0%

wave/network/static_handler.py                           34     34     10      0     0%

wave/testing/__init__.py                                  0      0      0      0   100%

wave/testing/devices_manager.py                          85     85     30      0     0%

wave/testing/event_dispatcher.py                        104    104     54      0     0%

wave/testing/results_manager.py                         494    494    240      0     0%

wave/testing/sessions_manager.py                        327    327    168      0     0%

wave/testing/test_loader.py                             145    145     98      0     0%

wave/testing/tests_manager.py                           261    261    144      0     0%

wave/testing/wpt_report.py                               24     24      8      0     0%

wave/utils/__init__.py                                    0      0      0      0   100%

wave/utils/deserializer.py                               89     89     46      0     0%

wave/utils/serializer.py                                 14     14      4      0     0%

wave/utils/user_agent_parser.py                          17     17      8      0     0%

wave/wave_server.py                                      50     50      6      0     0%

web_features/__init__.py                                  0      0      0      0   100%

web_features/manifest.py                                 68      3     14      1    95%

web_features/web_feature_map.py                          53      0     28      2    98%

webtransport/__init__.py                                  0      0      0      0   100%

webtransport/h3/__init__.py                               0      0      0      0   100%

webtransport/h3/capsule.py                               62     62     22      0     0%

webtransport/h3/handler.py                               13     13      0      0     0%

webtransport/h3/test_capsule.py                          98     77      4      2    23%

webtransport/h3/webtransport_h3_server.py               334    334     98      0     0%

---------------------------------------------------------------------------------------

TOTAL                                                  8883   5584   3388    173    36%



===== 892 passed, 12 skipped, 3 xfailed, 64 warnings in 303.92s (0:05:03) ======

py38: OK ✔ in 5 minutes 19.5 seconds
py38-flake8: install_deps> python -I -m pip install -r /home/test/web-platform-tests/tools/requirements_flake8.txt
py38-flake8: commands[0]> flake8 --append-config=/home/test/web-platform-tests/tools/flake8.ini
py38-flake8: OK ✔ in 7.52 seconds
py38-mypy: install_deps> python -I -m pip install -r /home/test/web-platform-tests/tools/requirements_mypy.txt
py38-mypy: commands[0] /home/test/web-platform-tests/tools/..> mypy --config-file=/home/test/web-platform-tests/tools/mypy.ini --explicit-package-bases tools
Success: no issues found in 306 source files

  py38: OK (319.50=setup[14.10]+cmd[305.40] seconds)
  py38-flake8: OK (7.52=setup[1.26]+cmd[6.27] seconds)
  py38-mypy: OK (10.53=setup[3.16]+cmd[7.37] seconds)
  congratulations :) (337.62 seconds)
+ export TOXENV=py38
+ TOXENV=py38
+ cd /home/test/web-platform-tests/tools/ci/../..
+ ./wpt test-jobs --includes wptrunner_unittest
+ cd tools/wptrunner
+ run_applicable_tox
+ local OLD_TOXENV=py38
+ unset TOXENV
++ tox -l
++ grep '^py38\(\-\|$\)'
++ tr '\n' ,
+ local RUN_ENVS=py38-base,
+ [[ -n py38-base, ]]
+ tox -e py38-base,
py38-base: install_deps> python -I -m pip install -r /home/test/web-platform-tests/tools/wptrunner/../requirements_pytest.txt -r /home/test/web-platform-tests/tools/wptrunner/requirements.txt
.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg: _optional_hooks> python /home/test/.local/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /home/test/.local/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_wheel> python /home/test/.local/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: prepare_metadata_for_build_wheel> python /home/test/.local/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /home/test/.local/lib/python3.8/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py38-base: install_package> python -I -m pip install --force-reinstall --no-deps /home/test/web-platform-tests/tools/wptrunner/.tox/.tmp/package/1/wptrunner-1.14.tar.gz
py38-base: commands[0]> pytest -c/home/test/web-platform-tests/tools/wptrunner/../pytest.ini --rootdir=/home/test/web-platform-tests/tools/wptrunner
============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-8.3.2, pluggy-1.5.0

cachedir: .tox/py38-base/.pytest_cache

rootdir: /home/test/web-platform-tests/tools/wptrunner

configfile: ../pytest.ini

plugins: hypothesis-6.112.0, cov-5.0.0

collecting ... 
collecting 157 items                                                           
collected 339 items                                                            



wptrunner/formatters/tests/test_chromium.py ...................          [  5%]

wptrunner/tests/browsers/test_base.py .                                  [  5%]

wptrunner/tests/browsers/test_sauce.py ..........                        [  8%]

wptrunner/tests/browsers/test_webkitgtk.py ssssssssssssssssssss          [ 14%]

wptrunner/tests/test_executors.py ........                               [ 17%]

wptrunner/tests/test_expectedtree.py .....                               [ 18%]

wptrunner/tests/test_formatters.py .....                                 [ 20%]

wptrunner/tests/test_manifestexpected.py ..........                      [ 23%]

wptrunner/tests/test_manifestupdate.py .                                 [ 23%]

wptrunner/tests/test_metadata.py ............                            [ 26%]

wptrunner/tests/test_products.py ssssssssssssssssssss................... [ 38%]

.ssssssssssssssssssss                                                    [ 44%]

wptrunner/tests/test_stability.py ......                                 [ 46%]

wptrunner/tests/test_testloader.py ........                              [ 48%]

wptrunner/tests/test_update.py ......................................... [ 60%]

...............                                                          [ 65%]

wptrunner/tests/test_wptrunner.py .                                      [ 65%]

wptrunner/tests/test_wpttest.py .........                                [ 68%]

wptrunner/wptmanifest/tests/test_conditional.py ........                 [ 70%]

wptrunner/wptmanifest/tests/test_parser.py .........                     [ 73%]

wptrunner/wptmanifest/tests/test_serializer.py ......................... [ 80%]

..................                                                       [ 85%]

wptrunner/wptmanifest/tests/test_static.py ......                        [ 87%]

wptrunner/wptmanifest/tests/test_tokenizer.py .......................... [ 95%]

................                                                         [100%]



======================= 279 passed, 60 skipped in 11.53s =======================

  py38-base: OK (21.20=setup[8.81]+cmd[12.40] seconds)
  congratulations :) (21.27 seconds)
+ export TOXENV=py38
+ TOXENV=py38
+ cd /home/test/web-platform-tests/tools/ci/../..
[taskcluster 2024-11-27 19:38:49.958Z] === Task Finished ===
[taskcluster 2024-11-27 19:38:50.129Z] Successful task run with exit code: 0 completed in 411.353 seconds