Skip piping non-code cells

[rijobro]

To autofix notebooks, I run a command like this:

jupytext "$filename" \
        --pipe "autoflake --in-place --remove-unused-variables --imports numpy,monai,matplotlib,torch,ignite {}" \
        --pipe "isort -" \
        --pipe "black -l 79 -" \
        --pipe autopep8 \
        --pipe "sed 's/ = list()/ = []/'"

This is removing spaces from the end of markdown lines, where a double space is needed to add a paragraph break.

Is there any way to prevent these autofixes from being applied to anything other than python cells?

[rijobro]

I'm not sure if this can be solved with --pipe-fmt. I can't see a list of arguments for this option, I've only seen auto:percent and py:percent. I'm not sure what the latter does, but it doesn't solve my problem.

[mwouts]

Hi @rijobro , thanks for sharing your use case!

This is removing spaces from the end of markdown lines, where a double space is needed to add a paragraph break.

Very interesting! Yes sure the Python-oriented tools don't know that the comment is written in Markdown, and thus they don't
respect this specificity.

Is there any way to prevent these autofixes from being applied to anything other than python cells?

Not at the moment. --pipe runs the selected tool on the text representation of the full notebook (in the format --pipe-fmt, which as you saw defaults to the percent format, i.e. the one with explicit # %% cell markers - probably the most robust format for this use case).

But I agree, most of the tools above will only act on the code, so it would make sense to allow running them only on the code cells. Would a --pipe-code option be what you're after? If you are open to making a PR, here a a few pointers. I think we could:

• (in the --pipe-code mode), make a copy of the original notebook that contain only the code cells before exporting it to the text buffer:
  

  jupytext/jupytext/cli.py

  Line 922 in 41b7e9c

  text = writes(notebook, fmt)

• and also, when we override the inputs of the original notebook with the reformatted ones at
  

  jupytext/jupytext/cli.py

  Line 972 in 41b7e9c

  piped_notebook = combine_inputs_with_outputs(piped_notebook, notebook, fmt)

  
  we should tell to the combine_inputs_with_outputs function that we want to preserve the non-code cells.

[rijobro]

Thanks for the quick reply!

Your suggested solution certainly looks like it would solve the problem. Unfortunately, I don't think I'll have time to implement it. In the meantime I've opted for a quick-and-easy fix: --pipe "autopep8 - --ignore W291".

This won't remove any EOL spaces for comments, which will include markdown and non-markdown, but it's a liveable solution for the time being.