Support for Julia language?

[dldx]

Hi,

This seems to be a very useful tool! Thanks for developing it! How much effort would it be to get it working for Julia as well? I wouldn't mind trying to add support if it isn't a huge amount of work.

Cheers,
Durand

[grst]

maybe maintan compatibility with http://weavejl.mpastell.com/stable/

[mwouts]

Thanks @dldx for suggesting! And @grst for the background on Julia markdown.

I expect that developping support for Julia markown should be relatively easy. Julia markdown would require

• a parser for markdown cell options
• a modification to the CellReader class accordingly (search for .Rmd and .md patterns)
• and to the CellExporter class
  as well as adding the .jmd extension in the jupytext.py module.

As @grst suggest, a good test for Julia markdown would be a comparison against weavejl.

But you may be more interested in supporting Julia scripts? In that case, the difficult part is to split scripts into cells without breaking functions or classes. For Python it was just a matter of finding whether the next line is indented or not, and make sure the current line is not in a multiline comment. In Julia is it easy to determine which blank line do really separate two code blocks? If you can solve that, then adding support for Julia should be feasible - and I will be glad to help you integrate that in jupytext!

[dldx]

Thanks for the info! In Julia, I imagine we can separate functions and structs by splitting on the "end" keyword but hmm, that could get pretty complicated with nested code blocks. Need to put a bit more thought into this...

I just realised that I have something more urgent to work on for the next couple of weeks but I will definitely come back to this if no one else has!

[mwouts]

No problem - I like the suggestion anyway. I had a look at a few Julia examples. At first sight it seems the Python parser would just work fine there (split blocks on blank lines that don't belong to multiline strings, and are not followed by indented code). Would you like to point us to a sample Julia notebook that you would like to appear in the tests?

[dldx]

Yeah, aside from indentation, I'd say that Julia syntax is pretty similar to Python. And since you managed to get R code working (and R generally has plenty of nested functions) with jupytext, Julia should also be achievable.

These examples are pretty simple, yet cover quite a few different cases. This one is about Julia itself so could be a good option.

[mwouts]

Thanks @dldx . Well, the notebooks I see should work perfectly with the Python parser indeed. One last question: I guess you are mostly interested in the export/import to Julia scripts, rather than Julia markdown - is that correct ? (I like the Julia markdown format, but since options have to be coded in Julia language, supporting it is a bit more involved...)

[dldx]

I am very interested in the ability to do paired code/notebook programming so yes, just to julia scripts and back :)

[mwouts]

@dldx , I have added support for Julia script in branch v0.6.2. Would you like to test and report whether it works as expected? Thanks!

[dldx]

Amazing! I tested it a little and it seems to be 90% there. I had one issue converting from .jl to ipynb.

After converting a julia file to ipynb, it thinks that a particular commented out part of code is actually a markdown cell. I suppose this is probably expected behaviour?

This was all commented out:

# This transforms a reactive signal into a resulting graph
# map(value -> interpolate_stations(all_station_data, value), year_signal)


# for i in 1941:10:2011
#        push!(year_signal, i)
#        sleep(2)
# end
#
# Plots.heatmap(max(rotl90(imresize(elevation_data, div.(size(elevation_data), 2))), 0), aspect_ratio=1)
# Plots.scatter!(coords[:col]./2, (size(elevation_data)[2] - coords[:row])./2)

and it turned into two different markdown cells rather than a commented out codeblock.

To be honest, this is fine. All I need to do is adjust the code slightly to make it look correct and it works perfectly! Thanks a lot!

[mwouts]

You're welcome! Yes, commented paragraphs are markdown. It's possible to get them as raw cells, though. To find the adequate syntax, open the file with Jupyter (using our ContentsManager), change the cell type to 'raw', save. Please report here!

Also, if you wanted to add more tests on Julia support, you're welcome. Just submit a PR on https://github.com/mwouts/jupytext/blob/v0.6.2/tests/test_read_simple_julia.py !

[mwouts]

By the way: if you want commented code cells (rather than markdown, or raw cells as I first suggested), you just need explicit code cell markers:

# + {}
# This transforms a reactive signal into a resulting graph
# map(value -> interpolate_stations(all_station_data, value), year_signal)
# -