Dev-Call "Testing Roadmap"

[freimair]

Agenda

• please make yourself familiar with the current state of the proposal
• please make yourself familiar with the testing tools proposal we agreed on in cycle 4
• general discussion
  • any thoughts? questions? request for change?
  • (@christophsturm) use Kotlin for tests? (convert source code to kotlin bisq#3026, Apply kotlin plugin and convert one unused class to kotlin bisq#3115)
  • Christoph showed us a work-in-progress test class he just is in the progress of converting to kotlin, core points:
    • the boilerplate overhead is significantly less than with Java
    • thus, make stuff easier to read and understand
    • lines of code are generally more efficient than plain Java
    • we believe we can save around 50% on LOC
  • there are core-contributors with at least some Kotlin experience
  • strategy is to let people proceed with creating tests (tests only!) with Kotlin, so people get a feel for Kotlin and after some time (weeks or even months), we may talk about getting Kotlin into the functional Bisq code
• discuss short-term strategy
  • any thoughts? questions? request for change?
  • how to get started
    • (@blabno) existing work
    • use queued up HTTP API?
    • create something new?
  • no detailed failure description is necessary, a simple yes/no is enough to see if a PR is good or not
  • best way would be to use the Java API provided by bisq core. However, the GUI (and the HTTP API) has business logic needed for Bisq to work properly
  • identifying/moving the logic from GUI to core is too much effort to get done for short-term effects
  • @blabno showed us some end-to-end tests that spin up Alice, Bob, an Arbitrator, a Seed Node and a Bitcoin node
  • using the queued up HTTP API seems to be the fastest way of getting short-term end-to-end testing up and running
• discuss long-term strategy
  • full TDD vs. test features vs. 2+2 or what if testing the changes in a PR would require massive resources (and therefore would delay a desperately needed fix)
    • because there haven't been any tests in place before?
    • because the feature touches the "heart" of Bisq and testing that would require a fairly high test coverage of Bisq to start with (which we do not have yet)
    • eg. the fix/optimization/countermeasure is against a corner case where the network has to be in a certain rare state and we are not sure how we can reproduce this state or would require loads of clients
  • if there is a bug, create a test that triggers the bug, then fix the code
  • nasty bugs may show themselves as a result of a very specific environment. In the case of Bisq, the environment is the highly dynamic P2P network, the bitcoin network, server performance (issues) and so forth. Recreating such an environment to trigger a bug might (at least) be difficult (if not impossible) and would take lots of resources.
  • human testing following a testing script. we have such a procedure (contact @devinbileck) but do not perform that on each and every release because it is so much effort. Yet, there are cases where a human tester can instantly spot an issue, where spotting that very issue (and others) is in fact not possible with automated testing
    • Maybe a manual testing script could be gradually refined to the extent that it could be coded, instead of risking coding integration tests before having a solid plan.
  • how to get started
    • migrating to the agreed tools (@christophsturm already did some valuable work on upping the test coverage)
    • only accept PRs where the introduced changes are tested?
  • any thoughts? questions? request for change?
    • do testing where it is feasible
    • keep the testing efforts reasonable

summing up

Schedule

When: 2019-08-22 5:30PM CEST
Duration: 1,5h
How: https://zoom.us/j/428072472, web link https://zoom.us/wc/join/428072472?pwd=

Video

https://youtu.be/n3kWsRAe2qk

[m52go]

Call recording is available here.