find_project_manifest raising errors when attempting to use quarto

[jmuchovej]

👋 first – thanks for your work on this! this definitely made switching to pixi much simpler than it would've otherwise been. 😄

i've been playing around with quarto lately, trying to use it instead of ipynbs. however, when i run quarto check juypter (with a pixi environment that indeed has jupyter), i appear to get errors from pixi-kernel of the variety:

Quarto 1.5.57
[✓] Checking Python 3 installation....OK
      Version: 3.12.5 (Conda)
      Path: /workspace/.pixi/envs/analysis/bin/python
      Jupyter: 5.7.2
      Kernels: pixi-kernel-xcpp14, pixi-kernel-python3, pixi-kernel-ir, pixi-kernel-xcpp11, pixi-kernel-bash, pixi-kernel-xcpp17, python3

(-) Checking Jupyter engine render....[pixi-kernel]: found Pixi 0.28.2
[pixi-kernel]: found empty project at /tmp/quarto-sessione0fecd1f23658c1e/37f126bda7102a58: PixiInfo(environments_info=[], project_info=None)
[pixi-kernel]: found empty project at /tmp/quarto-sessione0fecd1f23658c1e/37f126bda7102a58: PixiInfo(environments_info=[], project_info=None)
[pixi-kernel]: found empty project at /tmp/quarto-sessione0fecd1f23658c1e: PixiInfo(environments_info=[], project_info=None)
[pixi-kernel]: found empty project at /tmp/quarto-sessione0fecd1f23658c1e: PixiInfo(environments_info=[], project_info=None)
[pixi-kernel]: found empty project at /tmp: PixiInfo(environments_info=[], project_info=None)
[pixi-kernel]: found empty project at /tmp: PixiInfo(environments_info=[], project_info=None)
[pixi-kernel]: found empty project at /: PixiInfo(environments_info=[], project_info=None)
[pixi-kernel]: found empty project at /: PixiInfo(environments_info=[], project_info=None)
(\) Checking Jupyter engine render....[pixi-kernel]: No such comm target registered: quarto_kernel_setup

...

----- stderr -----
Pixi Kernel could not find a project manifest file in the current working directory /tmp/quarto-sessione0fecd1f23658c1e/37f126bda7102a58 nor in any of its parents.
Make sure you initialize a Pixi project by running 'pixi init' in the project directory and restart your kernel.

If you continue to face issues, please report them at https://github.com/renan-r-santos/pixi-kernel/issues
------------------

so, before claiming this is an issue with pixi-kernel, i'm curious: what is find_project_manifest aiming to do? i lightly read the code and wasn't able to infer what's going on, so i'm hoping you might be able to clarify.

aside: this is a project i'm running in a dev container – so it's possible some of these errors are dev container idiosyncrasies – but at face-value is seems like the error stems from quarto trying to work in a temporary file tree, rather than the project-local file tree.

[renan-r-santos]

Hi @jmuchovej, thank you for your detailed report. I'm not familiar with quarto, but I tried using it with pixi-kernel and it seemed to have worked fine. Here is my setup and log output

# pixi.toml
[project]
authors = []
channels = ["conda-forge"]
description = "Add a short description here"
name = "quarto"
platforms = ["osx-arm64"]
version = "0.1.0"

[dependencies]
quarto = ">=1.5.57,<2"
pixi-kernel = ">=0.4.0,<0.5"
jupyterlab = ">=4.2.5,<5"

Note that I installed quarto with Pixi too. With that, I get

pixi run quarto check jupyter                   
Quarto 1.5.57
[✓] Checking Python 3 installation....OK
      Version: 3.12.6 (Conda)
      Path: /Users/rrodrigues/Documents/coding/quarto/.pixi/envs/default/bin/python
      Jupyter: 5.7.2
      Kernels: pixi-kernel-xcpp17, pixi-kernel-xcpp11, python3, pixi-kernel-ir, pixi-kernel-bash, pixi-kernel-xcpp14, pixi-kernel-python3

[✓] Checking Jupyter engine render....OK

Then, if I run pixi run jupyter lab --notebook-dir ../myproject where myproject is a different folder where I have the following Pixi project

[project]
authors = []
channels = ["conda-forge"]
description = "Add a short description here"
name = "myproject"
platforms = ["osx-arm64"]
version = "0.1.0"

[dependencies]
ipykernel = ">=6.29.5,<7"
serialite = ">=0.3.4,<0.4"

I can create a notebook and use quarto render and preview commands:

Documents/coding/myproject quarto preview hello.ipynb
pandoc 
  to: html
  output-file: hello.html
  standalone: true
  section-divs: true
  html-math-method: mathjax
  wrap: none
  default-image-extension: png
  
metadata
  document-css: false
  link-citations: true
  date-format: long
  lang: en
  
Output created: hello.html

Watching files for changes
Browse at http://localhost:7641/

Could you post all the steps it takes to reproduce the issue you're reporting?

Regarding find_project_manifest, it looks for the nearest pixi.toml or pyproject.toml starting from your current working directory, usually the directory where you have your notebook opened. If it can't find one of these files, it goes up one directory and tries again or fails if it is in the root of the file system.
If it finds your project manifest, it will use that Pixi project environment to run your Notebook.

[jmuchovej]

👋 Sorry for the slow reply on this! Just circling back – I believe a the crux of this stems from not being able to install quarto from conda.

There's an open PR to add arm64 support for Linux here: conda-forge/quarto-feedstock#43, but it seems that its a problem further upstream with deno.

I'm gonna keep poking around to figure out how I might be able to get a system-installed quarto to play nicely with pixi-kernel.

Further, I suspect that part of the problem is that the system-installed quarto is starting from a true /tmp directory, while the conda installed one you tested with probably starts around .pixi/.

[jmuchovej]

👀 So I just upgraded pixi-kernel to 0.5.1 and it all seems to be working, even with a system installed quarto (though I had to use the default environment – which seems like expected behavior given #20).

(I suspect this is due to the refactor, which looks pretty promising! 🙂)

[renan-r-santos]

Nice to hear it is working fine with the new pixi-kernel. I'll close this issue but feel free to reopen it if you encounter other issues.