Merge pull request #87 from mrc-ide/develop

Minor tweaks to website

_freeze/website_docs/software_standards/execute-results/html.json

@@ -2,9 +2,7 @@
   "hash": "db135b5feb6049541afb42c37d885c0b",
   "result": {
     "markdown": "---\ntitle: \"Software standards\"\nformat: html\n---\n\n\n\n\n\n\n## Framework for evaluating software standards in *Plasmodium* genomics\n\nPGEforge aims to foster an ecosystem of high-quality, user-friendly tools that can be seamlessly integrated into genomic analysis workflows. One of the biggest challenges is the variability and lack of systematic assessment of existing tools, which often do not adhere to best practices in software development, including [FAIR standards](https://doi.org/10.1038/sdata.2016.18), maintenance, and usability.\n\nWorking towards this goal, a robust software standards evaluation framework was formulated to guide the development and assessment of tools used in *Plasmodium* genomic data analysis from both the end-user and developer perspective. This framework is crucial in addressing the variability and challenges associated with existing software tools but also to guide development of new tools, ensuring that they meet high standards of usability, accessibility, and reliability. \n\n### 'Ideal' software practices\nOne of the primary objectives of this framework is to define ‘ideal’ software practices that are not tool-specific but applicable across a range of genomic analysis tools. These practices encompass:\n\n- Comprehensive documentation\n- Ease of installation\n- Reliable and maintainable software\n\nAdditionally, during the [2023 RADISH23 hackathon](radish23.qmd), focused discussions highlighted the following software practices that are not necessarily essential, but are \"nice-to-have's\": \n\n- Uses standard data input formats \n- Computationally efficient\n- Informative error handling\n- Multiple languages for tutorials\n- Minimal dependencies\n- Modular code (eg split into functions)\n- Well annotated code\n\nThese practices can guide development of new tools and/or improvement of existing tools. \n\n### Evaluation criteria \nTo implement these standards and facilitate tool evaluation and development, PGEforge has developed a set of measurable criteria that can be applied to evaluate the performance and usability of various tools. There are two categories:\n\n- **User-facing:** criteria to evaluate the tool from an end-user perspective, for example whether installation instructions are available and easy-to-follow\n- **Developer-facing:** criteria to evaluate the tool from a developer perspective, for example whether unit tests are implemented\n\n::: {.column-margin}\nThe evaluation criteria encompass the following key themes, in line with the 'ideal' software practices for both end-users and developers:\n\n- Quality and comprehensiveness of documentation\n- Simplicity of installation processes\n- Quality assurance and maintenance\n:::\n\n\n::: {.cell}\n::: {.cell-output-display}\n`````{=html}\n<table class=\"table\" style=\"\">\n <thead>\n  <tr>\n   <th style=\"text-align:left;\"> Criteria </th>\n   <th style=\"text-align:left;\"> Type </th>\n   <th style=\"text-align:left;\"> Notes </th>\n  </tr>\n </thead>\n<tbody>\n  <tr grouplength=\"4\"><td colspan=\"3\" style=\"border-bottom: 1px solid;\"><strong>User-facing</strong></td></tr>\n<tr>\n   <td style=\"text-align:left;padding-left: 2em;\" indentlevel=\"1\"> Installation instructions exist </td>\n   <td style=\"text-align:left;\"> binary </td>\n   <td style=\"text-align:left;\">  </td>\n  </tr>\n  <tr>\n   <td style=\"text-align:left;padding-left: 2em;\" indentlevel=\"1\"> Usage instructions exist </td>\n   <td style=\"text-align:left;\"> binary </td>\n   <td style=\"text-align:left;\">  </td>\n  </tr>\n  <tr>\n   <td style=\"text-align:left;padding-left: 2em;\" indentlevel=\"1\"> Tutorials exist </td>\n   <td style=\"text-align:left;\"> binary </td>\n   <td style=\"text-align:left;\"> Must have explanation of inputs, outputs, and test data set in a worked example </td>\n  </tr>\n  <tr>\n   <td style=\"text-align:left;padding-left: 2em;\" indentlevel=\"1\"> Test data sets and results available </td>\n   <td style=\"text-align:left;\"> binary </td>\n   <td style=\"text-align:left;\">  </td>\n  </tr>\n  <tr grouplength=\"4\"><td colspan=\"3\" style=\"border-bottom: 1px solid;\"><strong>Developer-facing</strong></td></tr>\n<tr>\n   <td style=\"text-align:left;padding-left: 2em;\" indentlevel=\"1\"> Open source </td>\n   <td style=\"text-align:left;\"> binary </td>\n   <td style=\"text-align:left;\">  </td>\n  </tr>\n  <tr>\n   <td style=\"text-align:left;padding-left: 2em;\" indentlevel=\"1\"> Has software tests </td>\n   <td style=\"text-align:left;\"> binary </td>\n   <td style=\"text-align:left;\"> Eg, unit tests </td>\n  </tr>\n  <tr>\n   <td style=\"text-align:left;padding-left: 2em;\" indentlevel=\"1\"> More than 90% code coverage reported </td>\n   <td style=\"text-align:left;\"> binary </td>\n   <td style=\"text-align:left;\">  </td>\n  </tr>\n  <tr>\n   <td style=\"text-align:left;padding-left: 2em;\" indentlevel=\"1\"> Clear channels for software maintenance and issues </td>\n   <td style=\"text-align:left;\"> binary </td>\n   <td style=\"text-align:left;\"> Eg, GitHub issues, author contact information </td>\n  </tr>\n</tbody>\n</table>\n\n`````\n:::\n:::\n\n\n::: {.column-margin}\n<br>\n<br>\nEvery criteria is scored on the following scale: \n\n- 0: Criteria not fulfilled\n- 1: Criteria fulfilled but not entirely\n- 2: Criteria fulfilled \n\n<br>\nThis is then translated to an **end-user score** and **development score** for the tool. \n:::\n\n### Tool evaluation\nEvery *Plasmodium* genomic analysis tool can be evaluated against these objective software standards to provide these scores. The resulting evaluation matrix and overview of each tool can be found [here](tools_to_standards.qmd).\n\n",
-    "supporting": [
-      "software_standards_files"
-    ],
+    "supporting": [],
     "filters": [
       "rmarkdown/pagebreak.lua"
     ],

website_docs/future_work.qmd

@@ -21,18 +21,12 @@ By systematically benchmarking tools against known datasets, we can ensure that
 Ongoing work in this space includes **benchmarking COI estimation tools.**
 :::
 
-
 ## Adding and evaluating more tools
 Our initial efforts to [landscape available tools](tool_landscaping.qmd) were focused on tools for downstream *Plasmodium* genetic analysis and included 40 available tools. However, future work could include the following areas (not an exhaustive list!):
 
 - Tools with applications to mosquito genetics, as long as they are generally within the scope of PGEforge (i.e. *Plasmodium* genomic epidemiology tools)
 - Bioinformatic pre-processing tools/pipelines, such as variant calling and quality filters
 - Tools that are not specifically engineered for *Plasmodium* but can be applied to *Plasmodium*genetic data, such as more generic population genetics tools that estimate metrics such as F-statistics, extended haplotype homozygosity, etc
 
-## Tool development
-**Do we want to include some of the results from the tools to functionality evaluation? We could say here that tools that can phase are needed for all use cases!**
-
-Tools to functions -- we still can't phase! 
-
 ## Thank you! 
 Thank you for your interest in contributing to PGEforge. Your efforts help us build a stronger, more inclusive research community making *Plasmodium* genomic analysis accessible to all!

website_docs/how_to_contribute.qmd

@@ -8,12 +8,12 @@ format: html
 :::
 
 ## The PGEforge vision
-PGEforge is a **community-driven** platform created by and for malaria genomic epidemiology analysis tool developers and end-users. If you are interested in contributing to these efforts, please [get involved](how_to_contribute.qmd)!
+PGEforge is a **community-driven** platform created by and for malaria genomic epidemiology analysis tool developers and end-users. If you are interested in contributing to these efforts, please get involved!
 
 ## How to contribute to PGEforge
 We are excited to have you join us and we welcome input from all areas of the research community to continuously improve and expand our platform. PGEforge aims to contribute to an inclusive and sustainable ecosystem of existing and new tools for *Plasmodium* genomic data analysis. From less to more involvement, there are several ways to get involved and join us in contributing to these aims. All contributors must adhere to our [community rules](#community-rules) and will be [recognized as a contributor](contributors.qmd). 
 
-Below are some ways to get involved with current workstreams, but we also have some ideas and plans in the works for the [future](future_work.qmd) beyond what is currently available in PGEforge!
+Below are some ways to get involved with current workstreams, but we also have some ideas and plans in the works for the [future](future_work.qmd) beyond what is currently available in PGEforge! If you want to contribute in another way that is not currently listed, please [reach out to us directly](mailto:r.verity@imperial.ac.uk).
 
 ## Add tool to landscaping and software standards matrices
 Landscaping, documenting and benchmarking available *Plasmodium* genomic data analysis tools and any tools within the realm of *Plasmodium* genomic epidemiology (PGE) will require continuous updating and curation. As new tools become available, they need to be assessed and integrated into our existing framework. While we have made significant progress in identifying, documenting and evaluating tools commonly applied to *Plasmodium* genetic data, the [tool landscaping](tool_landscaping.qmd) and [evaluation against software standards](software_standards.qmd) is not exhaustive. For example, our initial efforts focused on tools for downstream *Plasmodium* genetic analysis, specifically targeting tools that extract signals from already processed data and focusing only on tools applicable *P. falciparum* and *P. vivax*. In terms of tool evaluation, we focused our efforts on evaluating the tools for which  resources were developed in the [tutorials section](tutorials_overview.qmd). [Future work](future_work.qmd) could also involve other types of tools, however.
@@ -72,9 +72,6 @@ You can find the templates for all of these documents in our [Github repository]
 
 **Please make sure to follow our [GitHub contribution guidelines](#contributing-guidelines-for-github).**
 
-## Developing analysis workflows
-Coming soon!
-
 ## Add yourself as a contributor
 Everyone who contributes to PGEforge is listed as a [contributor](contributors.qmd).