adopt a new format for pkgs/misc/vim-plugins/vim-plugin-names and maintainers/scripts/luarocks-packages.csv

[teto]

Issue description

I am trying to improve the way neovim plugins are handled. Traditionnally vim plugins are written in vimscript which lacks a dependency management system and so vim plugins tended to have no dependencies. There is now a vibrant community writing lua plugins which is a language with better tooling, notably luarocks that allows to specify dependencies in a rockspec file, for instance https://github.com/lewis6991/gitsigns.nvim/blob/main/gitsigns.nvim-scm-1.rockspec#L17.

I've made substantial changes to our lua infrastructure so we can leverage the rockspec in git repos directly (bypassing luarocks.org) in #134336.

The format of maintainers/scripts/luarocks-packages.csv has grown into a less practical csv file with format
name,src,ref,server,version,luaversion,maintainers
One of the appeal of using csv was that one could comment lines but now that the updater uses the python csv reader, it's not straightforward anymore (the python updater should be changed), also #134336 (comment) mentioned it was not ideal.

These new neovim plugins tend to be more neovim plugins and so we may need to think where to register those packages, (in
pkgs/misc/vim-plugins/vim-plugin-names or in maintainers/scripts/luarocks-packages.csv ?).
Also pkgs/misc/vim-plugins/vim-plugin-names could use some additional fields like the maintainers one.
Overall it would be nice if both files could share a similar format. Some kind of jsonc (with comment support) maybe.

[figsoda]

something like <owner>/<repo> [from <provider>] [as <name>] [requires <dependencies>] could be easy to migrate to

• nvim-telescope/telescope.nvim requires plenary-nvim,popup-nvim
• my-gitlab-username/my-plugin.nvim from gitlab as the-actual-name requires plenary-nvim

[figsoda]

and I don't think vim and luarocks need to use the same format. for vim plugins luaversion is not needed and maintainers can be set in ovverides.nix

[cyntheticfox]

I've been using neovim for a while now and am really liking the Lua use. I'm starting to wonder though if neovim needs its own package infrastructure, separate from vimPlugins. I've been using Packer.nvim as a stopgap measure, and it definitely feels very un-Nix to do it this way.

While directly using luaPackages for the interface works, it feels clunky and inaccurate, since these packages are really neovim-specific.

In that regard, might it be useful to adopt a neovimPlugins interface? Based on how it sounds like luaPackages is using csv, that might be difficult to implement though. With the addition of dependency integration, jsonc seems like a reasonable choice, but I have to ask then, what is the tradeoff we're making then such that we don't just use nix files for that, either as a single file or as multiple? (I'm familiar with using Nix and nixpkgs, not with the inner workings of nixpkgs (trying to learn though))

[teto]

I've hit #166007 and would like to change the format. I would vote for a csv file:
repo_uri,branch,specific_rev,maintainers.
Problem with csv is that adding a column is a pain. I dont like json because no comment/repetitive.
What about toml ? It forces us to fetch a toml dependency (I hear newer python will have a toml library included in the batteries) but is the best of both worlds, and is used by poetry too.

Would also help me prototype #160484 that kinda works but I am looking for ways to integrate it more cleanly in the nixpks ecosystem.

[cyntheticfox]

Toml is definitely more flexible, though csv is easier (computationally) to parse. That being said, considering the use case is more of for ease of maintenance and readability while maintaining efficient storage, not really making is computationally fast (that's what binary formats are for).

I prefer the look of Toml to other formats myself, but given Nix has a built-in JSON parser (iirc), I'd be inclined to maybe recommend either testing to see if the build time is significantly different between those two, or writing in a format such as TOML that is also converted to JSON by the maintainer via a script (less maintainable, but possibly faster at build time)

[teto]

nix 2.7 also has builtins.fromTOML

[teto]

I've submitted #166941 to address this issue. I ended up using CSV since I liked having one line per plugin. The change should make it easier if we want to swap with another json or toml backend later