jupyter server crashes and fails to render files or file ops if a malformed pyproject.toml is found in a user's homedir

[shaneknapp]

Description

when a malformed pyproject.toml is located in the root of a user's homedir, jupyter server picks it up, tries to execute it and fails.

Reproduce

1. stick a malformed pyproject.toml in your root homedir:

[project]
name = "mypackage"
version = "0.0.1"
dependencies = [
    "pandas",

2. log in/open your singleuser server
3. you will now be unable to see any files in either tree or lab views, and you won't be able to open or save any notebooks and instead get the following error dialogue box:
   

no files visible:

versus what is actually there:

this was discovered after a student had the above pyproject.toml located in their root homedir and was unable to open/create/save any notebooks or access any of their files through the UI. files were still visible if a terminal window was opened.

Expected behavior

honestly, we were surprised that pyproject.toml was ingested during single user server launch. if this is expected behavior (which i don't feel it should be), then it should fail gracefully.

Context

jupyter_server                2.14.2

Troubleshoot Output

the output of `jupyter troubleshoot` is quite long, and i can provide relevant information if needed.

Command Line Output

there's nothing pertinent in the output of `jupyter lab --debug`.

Error reported in the server logs:

[W 2024-07-15 11:26:04.642 ServerApp] Cannot load /pyproject.toml: Invalid value (at end of document)
[E 2024-07-15 11:26:04.643 ServerApp] Uncaught exception GET /user/<USERNAME>/api/contents/Untitled1.ipynb?type=notebook&content=1&1721067964537 (10.24.0.13)
    HTTPServerRequest(protocol='https', host='staging.datahub.berkeley.edu', method='GET', uri='/user/<USERNAME>/api/contents/Untitled1.ipynb?type=notebook&content=1&1721067964537', version='HTTP/1.1', remote_ip='10.24.0.13')
    Traceback (most recent call last):
      File "/srv/conda/lib/python3.11/site-packages/jupyter_server/services/contents/handlers.py", line 155, in get
        self.contents_manager.get(
    TypeError: build_jupytext_contents_manager_class.<locals>.JupytextContentsManager.get() got an unexpected keyword argument 'require_hash'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/srv/conda/lib/python3.11/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
                 ^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupyter_server/auth/decorator.py", line 73, in inner
        return await out
               ^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupyter_server/services/contents/handlers.py", line 168, in get
        self.contents_manager.get(
      File "/srv/conda/lib/python3.11/site-packages/jupytext/contentsmanager.py", line 207, in get
        config = self.get_config(path, use_cache=content is False)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupytext/contentsmanager.py", line 587, in get_config
        config_file = find_global_jupytext_configuration_file()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupytext/config.py", line 326, in find_global_jupytext_configuration_file
        config_file = find_jupytext_configuration_file(config_dir, False)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupytext/config.py", line 344, in find_jupytext_configuration_file
        doc = tomllib.loads(stream.read())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 102, in loads
        pos = key_value_rule(src, pos, out, header, parse_float)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 326, in key_value_rule
        pos, key, value = parse_key_value_pair(src, pos, parse_float)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 369, in parse_key_value_pair
        pos, value = parse_value(src, pos, parse_float)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 616, in parse_value
        return parse_array(src, pos, parse_float)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 420, in parse_array
        pos, val = parse_value(src, pos, parse_float)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 649, in parse_value
        raise suffixed_err(src, pos, "Invalid value")
    tomllib.TOMLDecodeError: Invalid value (at end of document)
[W 2024-07-15 11:26:04.646 ServerApp] wrote error: 'Unhandled error'
    Traceback (most recent call last):
      File "/srv/conda/lib/python3.11/site-packages/jupyter_server/services/contents/handlers.py", line 155, in get
        self.contents_manager.get(
    TypeError: build_jupytext_contents_manager_class.<locals>.JupytextContentsManager.get() got an unexpected keyword argument 'require_hash'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/srv/conda/lib/python3.11/site-packages/tornado/web.py", line 1790, in _execute
        result = await result
                 ^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupyter_server/auth/decorator.py", line 73, in inner
        return await out
               ^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupyter_server/services/contents/handlers.py", line 168, in get
        self.contents_manager.get(
      File "/srv/conda/lib/python3.11/site-packages/jupytext/contentsmanager.py", line 207, in get
        config = self.get_config(path, use_cache=content is False)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupytext/contentsmanager.py", line 587, in get_config
        config_file = find_global_jupytext_configuration_file()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupytext/config.py", line 326, in find_global_jupytext_configuration_file
        config_file = find_jupytext_configuration_file(config_dir, False)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/site-packages/jupytext/config.py", line 344, in find_jupytext_configuration_file
        doc = tomllib.loads(stream.read())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 102, in loads
        pos = key_value_rule(src, pos, out, header, parse_float)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 326, in key_value_rule
        pos, key, value = parse_key_value_pair(src, pos, parse_float)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 369, in parse_key_value_pair
        pos, value = parse_value(src, pos, parse_float)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 616, in parse_value
        return parse_array(src, pos, parse_float)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 420, in parse_array
        pos, val = parse_value(src, pos, parse_float)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/srv/conda/lib/python3.11/tomllib/_parser.py", line 649, in parse_value
        raise suffixed_err(src, pos, "Invalid value")
    tomllib.TOMLDecodeError: Invalid value (at end of document)

[Zsailer]

@shaneknapp, looking at the stack trace, this appears to be a bug in jupytext. If you disable jupytext and try again, do you see this issue?

[shaneknapp]

version 1.16.3 indeed had a bug, but i believe that the behavior of jupyter-server still might need some looking in to.

mainly, should the server app even ingest pyproject.toml from a user's root directory? and if so, perhaps this functionality should be better documented and advertised?

[krassowski]

mainly, should the server app even ingest pyproject.toml from a user's root directory

It doesn't. It's jupytext which is searching for it's configuration, see references to find_jupytext_configuration_file and similar in your traceback.

[shaneknapp]

mainly, should the server app even ingest pyproject.toml from a user's root directory

It doesn't. It's jupytext which is searching for it's configuration, see references to find_jupytext_configuration_file and similar in your traceback.

i see... so this is actually 'working as intended' even though i don't particularly like how jupytext handles this situation.

i'll close this now, thanks for the comments!