Draft notes for our October meeting

docs/community/meetings/2024-10-10.md

@@ -0,0 +1,138 @@
+# 10 October 2024
+
+Attendance: 4 in person, 3 online
+
+## Workshop retrospective
+
+We recently ran an [Introduction to Debugging workshop](../training/debugging/README.md) at the SPECTRUM Annual Meeting 2024 (23-25 September).
+We began this meeting by reflecting on what went well, and what could be improved for future versions of this workshop.
+
+!!! tip "Key message"
+
+    Feedback on the workshop was generally positive, although we encountered some technical challenges on the day.
+
+- Some workshop attendees reported an interest in running this workshop at a future [WIMSIG](https://austms.org.au/special-interest-groups/wimsig/) event.
+
+- Shortly after the workshop, Emma McBryde informed us that she encountered a difficult coding problem and solved it quickly thanks to the knowledge and skills that she learned from our workshop.
+
+- While almost all of the attendees already had R and Python installed, in future iterations we should ensure that each attendee already has R and Python installed prior to the workshop.
+
+- Limited internet connectivity in the workshop venue meant that we had to ask participants to download the Python and R scripts ahead of time, so we added links to each script to the [Resources page](../training/debugging/resources.md).
+
+We concluded that this workshop is a good foundation for planning the **debugging and testing topical guides**.
+
+- [The Debugging Book](https://www.debuggingbook.org/) might be a useful reference.
+
+## Future training events
+
+Given the success of the Introduction to Debugging workshop, we asked participants if there were other topics or skills that they would like to learn in a workshop or similar type of event.
+
+!!! tip "Key message"
+
+    There was broad interest in a testing workshop, and we discussed how good coding practices, testing, and debugging are highly interrelated skills.
+
+Several attendees immediately nominated learning how to test their code as a highly desirable skill, and a logical follow-up to the debugging workshop.
+
+- Saras noted that it's a skill that needs to actively maintained, and that it becomes harder and harder to do well when you haven't done it for a while.
+
+- As Rob pointed out in his presentation at the SPECTRUM & SPARK Annual Meeting in 2023, most of the introductions and training guides only present trivial examples, and these are not sufficiently detailed or relevant for most researchers.
+
+    - We need detailed examples that are relevant to our research domain(s).
+      Obvious candidates include moderately complex compartmental models (both deterministic and stochastic), and agent-based models.
+
+## How do people use tests?
+
+Nick asked how other people introduced testing into code bases that are not structured as software packages.
+
+!!! tip "Key message"
+
+    Perhaps the most difficult challenge is deciding **what to test**.
+
+- Nick himself has implemented tests as a separate workflow in the [Test, Trace, Isolation and Quarantine (TTIQ) repository](https://github.com/njtierney/ttiq-simulation?tab=readme-ov-file#running-tests).
+
+- Rob mentioned that he uses Continuous Integration (on the University of Melbourne's [GitLab instance](https://gitlab.unimelb.edu.au/)) to automatically run tests for most of his research code.
+  For example, this has allowed him to rapidly [catch and identify errors](../case-studies/moss-pypfilt-earlier-states.md) in his `pypfilt` package.
+
+    - But as Nick pointed out, this approach relies on your code already being reproducible, and perhaps this is too much of a hurdle for most people?
+
+- Eamon explained that he typically begins by writing tests, and makes them a part of his (C++) build system.
+  He sets up his workflow so that doesn't allow running simulations or experiments unless all of the tests are successful.
+
+- Rob explained that he finds tests very useful for articulating his ideas about, and expectations of, his code.
+
+A related question was deciding **what to test?**
+It isn't necessarily useful to aim for 100% test coverage (meaning that every line of code is executed by at least one case).
+So how do you write "maximum value" tests?
+
+We all agreed that it can be very scary to think about having your code subject to scrutiny after publication.
+
+- A high-profile example is Keisha Prem's 20211 [social contact matrices paper](https://doi.org/10.1371/journal.pcbi.1009098).
+  This paper received a lot of attention and scrutiny, and [a correction was published](https://doi.org/10.1371/journal.pcbi.1012454) in 2024 that provided a corrected version of the code.
+
+- Saras mentioned that as a journal editor, whenever a reviewer remarked that a result or output in a paper "looked weird", the authors would investigate and respond that it was an error.
+
+- Rob mentioned that he encountered this situation (as a reviewer) earlier in the year.
+
+## Writing testable code
+
+!!! tip "Key message"
+
+    Attendees reflected on their experiences of testing and debugging.
+    A common finding was that there is an art to writing code that is **easy to test and debug**.
+
+As we highlight at the start of the [Writing code](../../guides/writing-code/README.md) guide, code should be **written for people to read**!
+
+This is not easy!
+Much like writing good research papers, this is a skill that takes time to learn, and it gets easier with practice.
+
+- Nick has given a talk, [practical functions](https://github.com/njtierney/funfun), about writing code to make it easier to debug and test.
+
+- Saras reflected on a project that began with a large amount of inherited code.
+  The first step was to reorganise the code in order to understand it, and to make it easier to test and debug.
+
+- Saras also shared a relevant quote from the UK Research Software Engineering conference:
+
+    > Typically we see of code as a snapshot of a paper, but we could instead consider the paper as a snapshot of the code.
+
+The code is the living project, the manifestation of what you've done.
+It is the ultimate "methods section", and should be written in a way that explains and clarifies what you've done **and why you've done it**.
+
+!!! info "Refactoring"
+
+    We can't expect to write "perfect" code, because our goals and expectations can change during the research process.
+    So we may need to step back and restructure our code at times — this is known as "refactoring".
+
+    This is the coding equivalent of rewriting text (e.g., in a paper) to improve its logical flow and clarity.
+
+## A "show and tell" event
+
+!!! tip "Key message"
+
+    There was unanimous agreement that a "show and tell" event, where people present examples of well-written code, debugging, testing, etc, would be a great way to **share knowledge and good practices**.
+
+    This could also be an "on ramp" for peer code review?
+
+Several attendees said they would love to see a talk about testing the sort of  code that researchers write — as opposed to general software — and finding a good balance for research code, which is often only intended for reuse in a rather narrow sense.
+
+- Nefel observed that (some) PhD students do discuss ideas about how to implement certain things in their code, and may even be doing some form of peer code review.
+  She remarked that peer code review during development sounds really cool and fascinating, but also terrifying to do in retrospect — there's an understandable feeling that you don't want to know about errors in retrospect!
+
+- Several attendees stress that making mistakes in your code is not rare at all, and we should normalise it to remove any sense of shame.
+
+- Eamon explained that he likes to write tests while writing code, and develops his projects iteratively, creating test scripts as he goes and retaining them for regression testing.
+
+- Saras responded that she wanted a "show and tell" where Eamon could demonstrate how his build process works.
+
+We ended with a brief discussion about how a "show and tell" event might be structured, and came up with the following suggestions:
+
+- A half-day or whole-day event would probably be more useful and effective than only running for 1-2 hours;
+
+    - We might to obtain a modest amount of funding, so that we could provide basic catering for attendees.
+
+- Give each attendee a short amount of time (perhaps 10-15 minutes?) to present some aspect of their work, coding practices, workflow, etc;
+
+    - This could be something that they found particularly valuable;
+
+    - Or it could be a request for suggestions/feedback about how to improve something — peer code review in miniature!
+
+- Finish by summarising common themes, issues, solutions, etc, and potentially identify some general guidelines for people to incorporate into their research.

docs/community/meetings/README.md

@@ -2,6 +2,8 @@
 
 This section contains summaries of each Community of Practice meeting.
 
+- [10 October 2024](2024-10-10.md): workshop retrospective, identify future workshops.
+
 - [12 September 2024](2024-09-12.md): test run of our Introduction to Debugging workshop.
 
 - [8 August 2024](2024-08-08.md): orientation guide planning.

mkdocs.yml

@@ -88,6 +88,7 @@ nav:
     - community/README.md
     - "Meetings":
       - community/meetings/README.md
+      - "10 October 2024": community/meetings/2024-10-10.md
       - "12 September 2024": community/meetings/2024-09-12.md
       - "8 August 2024": community/meetings/2024-08-08.md
       - "11 July 2024": community/meetings/2024-07-11.md