Normalize jupytext config options names (#754)

* jupytext.read and write accept a config argument * Settings from the config file have precedence over those in the notebook at save time * Comment on why we use the config when reading text notebooks * Normalize the Jupytext configuration options - default_jupytext_formats -> formats - default_notebook_metadata_filter -> notebook_metadata_filter - default_cell_metadata_filter -> cell_metadata_filter - default_cell_markers -> cell_markers Previous option names are supported but will cause a DeprecationWarning * Rename "default_jupytext_formats" to "formats" in the tests and documentation * Rename "default_notebook_metadata_filter" to "notebook_metadata_filter" in the tests and documentation * Rename "default_cell_metadata_filter" to "cell_metadata_filter" in the tests and documentation * Rename "default_cell_markers" to "cell_markers" in the tests and documentation * On the way to Jupytext 1.11.0 * Use nbformat>=5.1.2 in tests to get rid of the 'Sampling from a set deprecated' warning * 'formats' in jupytext.toml can be a string, a list or a dict prefix => format name * Issue FutureWarnings rather than DeprecationWarnings when obsolete config names are used * Add contributing.md to the documentation * Use notebook_extensions from the config file if set #746 * One more test on config & metadata filters * Test that deprecated options work but result in a warning * Version 1.11.0rc0 * Version 1.11.0
mwouts · Mar 18, 2021 · 0a8b180 · 0a8b180
1 parent e28497a
commit 0a8b180
Show file tree

Hide file tree

Showing 33 changed files with 415 additions and 240 deletions.
diff --git a/README.md b/README.md
@@ -88,7 +88,7 @@ see the [documentation](docs/config.md#per-notebook-configuration).
 
 with e.g. the following content:
 ```
-default_jupytext_formats = "ipynb,py:percent"
+formats = "ipynb,py:percent"
 ```
 see the [documentation](docs/config.md#configuring-paired-notebooks-globally).
 </details>

diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
@@ -1,6 +1,17 @@
 Jupytext ChangeLog
 ==================
 
+1.11.0 (2021-03-18)
+-------------------
+
+**Fixed**
+- The `jupytext.toml` config file can now be used together with the `jupytext` pre-commit hook ([#752](https://github.com/mwouts/jupytext/issues/752))
+- The `notebook_extensions` option of the `jupytext.toml` file now works ([#746](https://github.com/mwouts/jupytext/issues/746))
+
+**Changed**
+- The options in `jupytext.toml` where renamed to match the `jupytext` metadata in the text notebooks. One should now use `formats` rather than `default_jupytext_formats` and `notebook_metadata_filter` rather than `default_notebook_metadata_filter` ([#753](https://github.com/mwouts/jupytext/issues/753))
+
+
 1.10.3 (2021-03-07)
 -------------------
 

diff --git a/docs/config.md b/docs/config.md
@@ -50,22 +50,36 @@ If `JUPYTEXT_CEILING_DIRECTORIES` is defined, Jupytext will stop searching for c
 
 The examples below assume that you use a `.jupytext`, `jupytext.toml` or `.jupytext.toml` Jupyter configuration file in TOML format. If you use another extension, please adapt the examples. For instance, if you want to use `jupytext.yml` in YAML format, replace the `=` sign with `:` and remove the double quotes. See also [`test_config.py`](https://github.com/mwouts/jupytext/blob/master/tests/test_config.py) for short examples in all the supported formats.
 
+Also, the examples are for Jupytext 1.11.0 or later. If you are using an older version, you should consult the [previous version](https://github.com/mwouts/jupytext/blob/v1.10.3/docs/config.md#configuring-paired-notebooks-globally) of this documentation.
+
 Say you want to always associate every `.ipynb` notebook with a `.md` file  (and reciprocally). This is done by adding the following to your `jupytext.toml` or `.jupytext.toml` Jupyter configuration file:
 ```
 # Always pair ipynb notebooks to md files
-default_jupytext_formats = "ipynb,md"
+formats = "ipynb,md"
 ```
 
 If you prefer to use a default `ipynb` - `py:percent` pairing, then that would be:
 ```
 # Always pair ipynb notebooks to py:percent files
-default_jupytext_formats = "ipynb,py:percent"
+formats = "ipynb,py:percent"
+```
+or alternatively, using an explicit format list:
+```
+# Always pair ipynb notebooks to py:percent files
+formats = ["ipynb", "py:percent"]
 ```
 
 You can pair notebooks in trees with a `root_prefix` separated with three slashes, e.g.
 ```
 # Pair notebooks in subfolders of 'notebooks' to scripts in subfolders of 'scripts'
-default_jupytext_formats = "notebooks///ipynb,scripts///py:percent"
+formats = "notebooks///ipynb,scripts///py:percent"
+```
+or alternatively, using a dict to map the prefix path to the format name:
+```
+# Pair notebooks in subfolders of 'notebooks' to scripts in subfolders of 'scripts'
+[formats]
+"notebooks/" = "ipynb"
+"scripts/" = "py:percent"
 ```
 The `root_prefix` is matched with the top-most parent folder of the matching name, not above the Jupytext configuration file.
 
@@ -82,23 +96,23 @@ Please note that, while Jupytext is Jupyter acts accordingly to both local or gl
 
 ### Metadata filtering
 
-You can specify which metadata to include or exclude in the text files created by Jupytext by setting `default_notebook_metadata_filter` (notebook metadata) and `default_cell_metadata_filter` (cell metadata) in the configuration file. They accept a string of comma separated keywords. A minus sign `-` in front of a keyword means exclusion.
+You can specify which metadata to include or exclude in the text files created by Jupytext by setting `notebook_metadata_filter` (notebook metadata) and `cell_metadata_filter` (cell metadata) in the configuration file. They accept a string of comma separated keywords. A minus sign `-` in front of a keyword means exclusion.
 
 Suppose you want to keep all the notebook metadata but `widgets` and `varInspector` in the YAML header. For cell metadata, you want to allow `ExecuteTime` and `autoscroll`, but not `hide_output`. You can set
 ```python
-default_notebook_metadata_filter = "all,-widgets,-varInspector"
-default_cell_metadata_filter = "ExecuteTime,autoscroll,-hide_output"
+notebook_metadata_filter = "all,-widgets,-varInspector"
+cell_metadata_filter = "ExecuteTime,autoscroll,-hide_output"
 ```
 
 If you want that the text files created by Jupytext have no metadata, you may use the global metadata filters below. Please note that with this setting, the metadata is only preserved in the `.ipynb` file.
 ```python
-default_notebook_metadata_filter = "-all"
-default_cell_metadata_filter = "-all"
+notebook_metadata_filter = "-all"
+cell_metadata_filter = "-all"
 ```
 
 It is possible to filter nested metadata. For example, if you want to preserve the Jupytext metadata, but not the Jupytext version number, you can use:
 ```python
-default_notebook_metadata_filter = "-jupytext.text_representation.jupytext_version"
+notebook_metadata_filter = "-jupytext.text_representation.jupytext_version"
 ```
 
 Finally, to hide the notebook metadata in an HTML comment in Markdown files, use the option `hide_notebook_metadata`.

diff --git a/docs/contributing.md b/docs/contributing.md
@@ -0,0 +1,32 @@
+# Contributing
+
+Thanks for reading this. Contributions to this project are welcome.
+And there are many ways you can contribute...
+
+## Spread the word
+
+You like Jupytext? Probably your friends and colleagues will like it too.
+Show them what you've been able to do with: version control, collaboration on notebooks, refactoring of notebooks, notebooks integrated in library, notebook generated from Markdown documents...
+
+By the way, we're also interested to know how you use Jupytext! There may well be applications we've not thought of!
+
+## Improve the documentation
+
+You think the documentation could be improved? You've spotted a typo, or you think you can rephrase a paragraph to make is easier to follow? Please follow the _Edit on Github_ link, edit the document, and submit a pull request.
+
+## Report an issue
+
+You have seen an issue with Jupytext, or you can't find your way in the documentation?
+Please let us know, and provide enough information so that we can reproduce the problem.
+
+## Propose enhancements
+
+You want to submit an enhancement on Jupytext? Unless this is a small change, we usually prefer that you let us know beforehand: open an issue that describe the problem you want to solve.
+
+## Add support for another language
+
+A pull request for which you do not need to contact us in advance is the addition of a new language to Jupytext. In principle that should be easy - you would only have to:
+- document the language extension and comment by adding one line to `_SCRIPT_EXTENSIONS` in `languages.py`.
+- add the language to `docs/languages.md`
+- contribute a sample notebook in `tests/notebooks/ipynb_[language]`.
+- run the tests suite (with just `pytest`). The mirror tests will generate various text representations corresponding to your notebook under  `tests/notebooks/mirror/`. Please verify that these files are valid scripts, and commit them.
diff --git a/CONTRIBUTING.md → docs/developing.md b/CONTRIBUTING.md → docs/developing.md
@@ -1,46 +1,21 @@
-# Contributing to Jupytext
+# Developing Jupytext
 
-Thanks for reading this. Contributions to this project are welcome.
-And there are many ways you can contribute...
-
-## Spread the word
-
-You like Jupytext? Probably your friends and colleagues will like it too.
-Show them what you've been able to do with: version control, collaboration on notebooks, refactoring of notebooks, notebooks integrated in library, notebook generated from Markdown documents...
-
-By the way, we're also interested to know how you use Jupytext! There may well be applications we've not thought of!
-
-## Improve the documentation
-
-You think the documentation could be improved? You've spotted a typo, or you think you can rephrase a paragraph to make is easier to follow? Please follow the _Edit on Github_ link, edit the document, and submit a pull request.
-
-## Report an issue
-
-You have seen an issue with Jupytext, or you can't find your way in the [documentation](https://jupytext.readthedocs.io)?
-Please let us know, and provide enough information so that we can reproduce the problem.
-
-## Test the development version
+## How to test development versions from GitHub
 
 If you want to test a feature that has been integrated in `master` but not delivered yet to `pip` or `conda`, use
 ```
 pip install git+https://github.com/mwouts/jupytext.git
 ```
 
-## Propose enhancements
-
-You want to submit an enhancement on Jupytext? Unless this is a small change, we usually prefer that you let us know beforehand: open an issue that describe the problem you want to solve.
-
-## Add support for another language
+If you want to test Jupytext in JupyterLab 3 then you will have to build the extension for JupyterLab. To do so, make sure that you have a recent version of `node`, and prefix the command above with `BUILD_JUPYTERLAB_EXTENSION=1`.
 
-A pull request for which you do not need to contact us in advance is the addition of a new language to Jupytext. In principle that should be easy - you would only have to:
-- document the language extension and comment by adding one line to `_SCRIPT_EXTENSIONS` in `languages.py`.
-- add the language to `docs/languages.md`
-- contribute a sample notebook in `tests/notebooks/ipynb_[language]`.
-- run the tests suite (with just `pytest`). The mirror tests will generate various text representations corresponding to your notebook under  `tests/notebooks/mirror/`. Please verify that these files are valid scripts, and commit them.
-
-# How to setup a development environment for Jupytext
+Finally, if you want to test a development branch, use
+```
+pip install git+https://github.com/mwouts/jupytext.git@branch
+```
+where `branch` is the name of the branch you want to test (again, prefix the command above with `BUILD_JUPYTERLAB_EXTENSION=1` if you want to use Jupytext within JupyterLab 3).
 
-## Jupytext as a Python package
+## Install and develop Jupytext locally
 
 Most of Jupytext's code is written in Python. To develop the Python part of Jupytext, you should clone Jupytext, then create a dedicated Python env:
 ```

diff --git a/docs/formats.md b/docs/formats.md
@@ -205,11 +205,11 @@ Metadata can be associated to a given cell using a key/value representation:
 
 The `light` format can use custom cell markers instead of `# +` or `# -`. If you prefer to mark cells with VS Code/PyCharm (resp. Vim) folding markers, set `"cell_markers": "region,endregion"` (resp. `"{{{,}}}"`) in the jupytext section of the notebook metadata. If you want to configure this as a global default, add either
 ```python
-c.ContentsManager.default_cell_markers = "region,endregion"  # Use VS Code/PyCharm region folding delimiters
+c.ContentsManager.cell_markers = "region,endregion"  # Use VS Code/PyCharm region folding delimiters
 ```
 or
 ```python
-c.ContentsManager.default_cell_markers = "{{{,}}}"           # Use Vim region folding delimiters
+c.ContentsManager.cell_markers = "{{{,}}}"           # Use Vim region folding delimiters
 ```
 to your `.jupyter/jupyter_notebook_config.py` file.
 
@@ -277,7 +277,7 @@ jupytext --update-metadata '{"jupytext": {"cell_markers": "\"\"\""}}' notebook.i
 
 If you want to use multiline comments for all your paired notebooks, you could also add
 ```python
-c.ContentsManager.default_cell_markers = '"""'
+c.ContentsManager.cell_markers = '"""'
 ```
 to your `.jupyter/jupyter_notebook_config.py` file.
 

diff --git a/docs/index.md b/docs/index.md
@@ -70,7 +70,7 @@ see the [documentation](config.md#per-notebook-configuration).
 
 with e.g. the following content:
 ```
-default_jupytext_formats = "ipynb,py:percent"
+formats = "ipynb,py:percent"
 ```
 see the [documentation](config.md#configuring-paired-notebooks-globally).
 </details>
@@ -114,5 +114,7 @@ using-library.md
 examples.md
 faq.md
 tutorials.md
+contributing.md
+developing.md
 CHANGELOG.md
 ```
diff --git a/docs/paired-notebooks.md b/docs/paired-notebooks.md
@@ -44,12 +44,6 @@ In JupyterLab this is slightly different. Scripts and Markdown document also hav
 ![](https://github.com/mwouts/jupytext-screenshots/raw/master/JupytextDocumentation/ContextMenuLab.png)
 
 If do not want to classify scripts or Markdown documents as notebooks, please use the `notebook_extension` option. For instance, if you want to get the notebook icon only for `.ipynb` and `.Rmd` files, set
-
-```python
-c.NotebookApp.contents_manager_class = "jupytext.TextFileContentsManager"
-c.ContentsManager.notebook_extensions = "ipynb,Rmd"
-```
-in your `.jupyter/jupyter_notebook_config.py`, or
 ```
 notebook_extensions = "ipynb,Rmd"
 ```

diff --git a/docs/using-pre-commit.md b/docs/using-pre-commit.md
@@ -16,7 +16,7 @@ You can provide almost all command line arguments to Jupytext in pre-commit, for
 ```yaml
 repos:
 -   repo: https://github.com/mwouts/jupytext
-    rev: v1.10.0  # CURRENT_TAG/COMMIT_HASH
+    rev: v1.11.0  # CURRENT_TAG/COMMIT_HASH
     hooks:
     - id: jupytext
       args: [--from, ipynb, --to, "py:percent"]
@@ -27,15 +27,15 @@ If you are combining Jupytext with other pre-commit hooks, you must ensure that
 ```yaml
 repos:
 -   repo: https://github.com/mwouts/jupytext
-    rev: v1.10.0  # CURRENT_TAG/COMMIT_HASH
+    rev: v1.11.0  # CURRENT_TAG/COMMIT_HASH
     hooks:
     - id: jupytext
       args: [--sync, --pipe, black]
       additional_dependencies:
-        - black==19.10b0 # Matches hook
+        - black==20.8b1 # Matches hook
 
 -   repo: https://github.com/psf/black
-    rev: 19.10b0
+    rev: 20.8b1
     hooks:
     - id: black
       language_version: python3

diff --git a/environment-ci.yml b/environment-ci.yml
@@ -8,7 +8,7 @@ dependencies:
   - jupyter
   - ipykernel
   - nbconvert
-  - nbformat>=5.1.1
+  - nbformat>=5.1.2
   - pyyaml
   - toml
   - markdown-it-py~=0.6.0

diff --git a/environment.yml b/environment.yml
@@ -5,7 +5,7 @@ channels:
 dependencies:
   - python>=3.6
   - jupyterlab>=3.0
-  - nbformat>=5.1.1
+  - nbformat>=5.1.2
   - jupyter-packaging
   - pyyaml
   - toml

diff --git a/jupytext/__init__.py b/jupytext/__init__.py
@@ -22,7 +22,7 @@ def _jupyter_labextension_paths():  # pragma: no cover
 
 def load_jupyter_server_extension(app):  # pragma: no cover
     """Use Jupytext's contents manager"""
-    if hasattr(app.contents_manager_class, "default_jupytext_formats"):
+    if hasattr(app.contents_manager_class, "formats"):
         app.log.info(
             "[Jupytext Server Extension] NotebookApp.contents_manager_class is "
             "(a subclass of) jupytext.TextFileContentsManager already - OK"