Testing infrastructure #1410
Assignees
Comments
This was referenced Oct 17, 2017
Closed
bors bot
added a commit
that referenced
this issue
Oct 19, 2017
1589: [RFC] Warden test framework r=msiglreith a=kvark This is a pre-alpha of the new test infrastructure. It's based on [RON](https://github.com/ron-rs/ron)-defined tests that are de-serialized and executed on each available native backend. See [readme](https://github.com/kvark/gfx/tree/warden/src/warden) for ~~devil in the~~ details. For Unix machines, all you need is `make reftests`. For Windows (without WSL): `(cd src/warden && cargo run --features vulkan,dx12,logger)` Output: ``` Warding Vulkan: AdapterInfo { name: "Radeon(TM) RX 460 Graphics ", vendor: 4098, device: 26607, software_rendering: false } Loading scene 'basic': Test 'render-pass-clear' ... ran: PASS Warding DX12: AdapterInfo { name: "Radeon(TM) RX 460 Graphics", vendor: 4098, device: 26607, software_rendering: false } Loading scene 'basic': Test 'render-pass-clear' ... ran: PASS ``` Fixes #1410 Note: it will really shine when: 1. we get GL support - #1539 2. hook Warden up with headless glutin context - #1410 3. run on Travis CI HAL changes: - `Instance` specifies `Backend` as an associated type - `create_renderpass` -> `create_render_pass` for naming consistency The PR has a lot of unfinished bits (e.g. PSO creation) but it's functional and technically ready for merging. I'm not 100% confident that being data-driven is a good thing here, hence `RFC` in the subject: - (+) RON is more expressive (maps, enum values, etc) - (+) somewhat consistent with [Wrench](https://github.com/servo/webrender/tree/master/wrench/reftests) - (-) RON is another format/language (note: can do any `serde`-supported format, really) - (~) run-time checked instead of compile-time. This is not really a great point since we are supposed to run those tests everywhere all the time, so we shouldn't end up in a situation where someone built them but didn't run.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Having head-less tests (#583) on CI is great, but that would only test GL backend and only on a single implementation.
We need a set of tests that the devs/users will run on their local machines to ensure the correctness of all backends supported by their machines. Thus, the tests are to be written in a backend-independent way, preferably constructed by a simple readable/writable configuration (in RON format?..). The harness would just run them through all available backends on the local machine, using head-less mode where possible.
Having such tests is crucial for us to keep features working consistently on the way to a fully featured multi-backend implementation.
The text was updated successfully, but these errors were encountered: