-
Notifications
You must be signed in to change notification settings - Fork 194
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Output schema documentation as Markdown #1381
Conversation
Codecov Report
@@ Coverage Diff @@
## dev #1381 +/- ##
==========================================
+ Coverage 66.72% 67.02% +0.30%
==========================================
Files 50 50
Lines 5692 5735 +43
==========================================
+ Hits 3798 3844 +46
+ Misses 1894 1891 -3
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Love it! Just tested locally and worked a treat.
Couple of minor requests:
- Can we get the resulting markdown to pass
markdownlint
if possible? Or at least these two rules:- Headings should be surrounded by blank lines
- There are some trailing spaces after the final column title:
Hidden
- Might be nice to use
inline code
backticks a bit more liberally around parameter names and types?
Great suggestions, @ewels! The latest commit should fix those. |
Nice! 😍 |
But it's not really exporting the schema? It's generating prose documentation from it. So I quite like the current command personally.. |
I guess |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great! Tried again and working really nicely now.
Two remaining things I've noticed on this round:
- The current code will ignore any parameters not within a
definition
.- For nf-core pipelines, everything should be here as we lint for it (I think?). But ungrouped parameters from
nf-core schema build
will just be in the top levelobject
and need to be handled separately. - You can see this logic in various bits of code that work with the schema, for example the website docs code.
- For nf-core pipelines, everything should be here as we lint for it (I think?). But ungrouped parameters from
- We're missing some schema fields
- Most notable is
help_text
. I appreciate that this is difficult to render in the markdown. Maybe there could be a section at the bottom of the document with all of the help text blocks and the table parameter name could link to the correct heading? - Also missing some of the more specialised fields, such as
pattern
,enum
,min
,max
. Again might make the table too verbose, so not sure what the best thing is to do about these.
- Most notable is
Minor things I wondered:
- Would be nice if we could specify the name / path to a pipeline as well as / instead of a path to a schema file. eg.
nf-core schema docs rnaseq
,nf-core schema docs nf-core/rnaseq
and so on. Some other commands work in this way I think so can hopefully reuse that code. - Might be nice to have a built-in functionality to output HTML snippets as well. It's pretty simple, see for example the pipeline
markdown_to_html.py
scripts. Would then need an additional flag for--format
(which can default to markdown).
The direction I was thinking of going in was to have the option to build HTML outputs using a Jinja2 template (with a default supplied or a user-supplied one optionally). Then we could get quite fancy with styling the output and essentially replicate a white-label version of the nf-core website if we want. But this starts to get substantially more complicated and may not be worth the effort. So I'm certainly happy with merge without any of that, can always add it at a later date.
To make it easier to work out of the box and make it again a bit more similar to the other nf-core commands, could we make For the Coming back to |
@ewels @mashehu Those suggestions sound good but I'm probably not the right person to attempt them right now, given that I haven't actually worked with nf-core pipelines before and I'm not familiar with any of those special constructs. I'll circle back to this after I have more experience, if no one else has picked it up -- I'm going to attempt to convert some of our Nextflow image analysis pipelines to nf-core at some point soon. |
Just made a start on some of the simpler bits here:
I haven't attempted any of the more difficult / important bits yet 😅 I will do if I get time. |
Ok, I think that this is good to merge now. Just added a couple more bits:
There's certainly more that we can do with this, but for now I think it's fine to merge it in and let people play with it. Can add new features to it if there's demand.. Many thanks @krokicki! Phil |
Failing tests are unrelated to this PR, see #1436 👉🏻 merging anyway. |
Per discussion in Slack and partially addressing #741, I added a
nf-core schema docs
command which takes any nextflow_schema.json and generates Markdown documentation for it.PR checklist
CHANGELOG.md
is updateddocs
is updatedExample Output (eager pipeline)
nf-core/eager pipeline parameters
A fully reproducible and state-of-the-art ancient DNA analysis pipeline
Input/output options
Define where the pipeline should find input data, and additional metadata.
input
string
udg_type
string
single_stranded
boolean
single_end
boolean
colour_chemistry
integer
bam
boolean