Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Player-based GPU testing #786

Closed
kvark opened this issue Jul 12, 2020 · 0 comments · Fixed by #803
Closed

Player-based GPU testing #786

kvark opened this issue Jul 12, 2020 · 0 comments · Fixed by #803
Assignees
Labels
type: enhancement New feature or request

Comments

@kvark
Copy link
Member

kvark commented Jul 12, 2020

Is your feature request related to a problem? Please describe.
This repository contains all the nasty implementation logic for WebGPU, yet there are no tests or examples here that allow us to verify that the logic is consistent, and new features work. We currently recommend preparing a wgpu-rs PR draft, so that implementors test its examples on the new code. This unfortunately adds complexity to our workflow, we'd like to make this as simple as cargo test in wgpu.

Describe the solution you'd like
We could have something along the lines of Warden in gfx land - gfx-rs/gfx#1589

Roughly speaking, that would be a set of RON-based recordings with some meta-data describing what the expectations are (outside of "should not error/panic"). For example, the contents of some texture at row X should be bytes A, B, C (see warden example).

The project layout could be the following:

  • player is turned from a binary app ("play") into a lib with a single binary. This means cargo run still works the same way. The logic of GPU operations go into the lib. Ideally, the logic of window handling would go into the binary.
  • player/tests folder is added. Each rust file in it would load a RON file describing a list of RON traces with the corresponding expectations. Not sure where we should put the two kinds of RON files. One possible location is in player/tests/traces and player/tests/sets, or something like that.
  • if a test fails to request adapters and devices, we issue a warning but do not fail the test. This will allow it to be run on any CIs without GPUs

As a result, cargo test should be doing a bunch of API replays on GPU and testing our logic.

Describe alternatives you've considered
I don't think there is another solution that fully covers the need here. But generally there are solid infrastructures at different levels that can partially help, like:

  • actual WebGPU CTS will be run on either NodeJS bindings to wgpu-native, or on Gecko
  • Vulkan CTS is ran on gfx-rs via gfx-portability

Additional context
It's soft-blocked on #792

I wonder if we can easily extend this to test for errors. I.e. verify that a particular scenario leads to some specific error.
It's not required here, since we are going to be tested by the upstream WebGPU conformance test suite, which has that.

@kvark kvark added the type: enhancement New feature or request label Jul 12, 2020
@kvark kvark pinned this issue Jul 12, 2020
@kvark kvark self-assigned this Jul 17, 2020
@bors bors bot closed this as completed in 2455a65 Jul 17, 2020
@kvark kvark unpinned this issue Jul 17, 2020
kvark added a commit to kvark/wgpu that referenced this issue Jun 3, 2021
786: Update to naga-13 r=kvark a=kvark



Co-authored-by: Dzmitry Malyshau <kvarkus@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant