noto-fonts: 20201206-phase3 -> 23.4.1

[jopejoe1]

Description of changes

Changes the GitHub repo to https://github.com/notofonts/notofonts.github.io and update the script to handle the new file structure and added my self as maintainer.

A list of updated fonts can be found at https://github.com/notofonts/notofonts.github.io/commits/NotoSerifKhitanSmallScript-v1.000/fonts and specific changes to one font can be found in the font specific repos at https://github.com/notofonts.

Some highlights include:

• Added fonts for Khitan-Small-Script
• Added Some Unicode 14.0 to NotoSans
• Split NotoSansSyriac into multiple fonts
• and more

After some changes, it now closes the following:
closes #170355
closes #166953

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[Mathnerd314]

Thanks for the PR, glad to see someone else is interested in maintaining these fonts.

https://github.com/notofonts/notofonts.github.io still says "WORK IN PROGRESS: [...] For now, please continue to go [...] to https://github.com/googlefonts/noto-fonts to download the entire collection." Particularly since some fonts are missing, I think using https://github.com/googlefonts/noto-fonts would be better.

I already made a PR to update the fonts, #170355, but it got stalled due to koreader. I think there are also some other questions like whether switching to variable fonts is a good idea, but I think those are best answered by committing it and seeing how many people complain. Up to you if you want to reuse my Python script for processing fonts, the approach here is a bit simpler.

Also style nit, https://nixos.org/manual/nixpkgs/unstable/#sec-package-naming says " If a package is not a release but a commit from a repository, [...] the date must be in unstable-YYYY-MM-DD format."

[jopejoe1]

Particularly since some fonts are missing,

From looking at the repos, there are indeed a few fonts that are missing that were in the old repo:

• Arimo, Cousine, Tinos (Not sure if these ones even should be in noto-fonts)
• Noto Sans Display (The source files for this were never in the latin-greek-cyrillic repo and the Serif Display files got removed from that repo, so not sure if we are ever going to see a new version of this font)
• Noto Kufi Arabic (This one is diffidently missing)

If a package is not a release but a commit from a repository, [...] the date must be in unstable-YYYY-MM-DD format."

Missed that, also missed typed the year gone, fix that with a rebase.

I already made a PR to update the fonts, #170355, but it got stalled due to koreader. I think there are also some other questions like whether switching to variable fonts is a good idea, but I think those are best answered by committing it and seeing how many people complain. Up to you if you want to reuse my Python script for processing fonts, the approach here is a bit simpler.

Gone look into that and see what's needed to change over to variable fonts, but probably need to mark koreader as broken for now if we change to variable fonts.

[jopejoe1]

Switched it over to variable fonts where possible and the ones were not I used otf fonts with the only exception being Noto Sans Math which only has a ttf font, also removed noto-fonts-extra as all weights are included in the variable font and the ones that don't have a variable font only have the regular versions.

[Mathnerd314]

* Arimo, Cousine, Tinos (Not sure if these ones even should be in noto-fonts)

Right, in the other PR I split those out into a package croscore-fonts. Google actually has separate source repos for them under https://github.com/googlefonts/

* Noto Sans Display (The source files for this were never in the latin-greek-cyrillic repo and the Serif Display files got removed from that repo, so not sure if we are ever going to see a new version of this font)

This is another weird one, the source files got merged with the main Noto Sans fonts. So you get the Display style of the Noto Sans font, rather than a separate Noto Sans Display font. I opened notofonts/notobuilder#17 but they haven't done anything yet.

[Mathnerd314]

Yeah I guess, it's been long enough that any update is worthwhile. I think we might get a fair number of complaints about the switch to variable fonts / otfs though. Something to consider for whoever actually merges this.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/1764

[jopejoe1]

https://github.com/notofonts/notofonts.github.io still says "WORK IN PROGRESS: [...] For now, please continue to go [...] to https://github.com/googlefonts/noto-fonts to download the entire collection." Particularly since some fonts are missing, I think using https://github.com/googlefonts/noto-fonts would be better.

Looks like they removed the work in progress section and archived the old repo.

[jopejoe1]

Updated to 23.2.1

[Mathnerd314]

LGTM.. Someone should probably test it on a real NixOS KDE system to make sure it works OK, but I'm lazy.

[jopejoe1]

Anything holding this getting merged back?

cc @laalsaas @emilazy

[Mathnerd314]

Still seems like a good idea, too bad about it being lost among the other 4k PR's.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/2052

[Mathnerd314]

For reference, if someone wants to package the croscore fonts, they are in these locations:

• https://github.com/googlefonts/Arimo/tree/main/fonts/vf
• https://github.com/googlefonts/cousine/tree/main/fonts/ttf/unhinted/variable_ttf
• https://github.com/googlefonts/tinos/tree/main/fonts/ttf/unhinted/variable_ttf

Not sure about unhinted vs hinted variable TTF, I think it's just Freetype's autohinting again. I'm not sure the autohinting is actually enabled by default on NixOS though so may be worth waiting for that to be resolved. (Edit: seems like it is, any non-hinted fonts are autohinted. The autohint=false config is disabling overriding the hinting of already-hinted fonts with the autohinting)

[alapshin]

According to #222311 both Qt5 and Qt6 don't support variable fonts well and require additional patches. For me rendering of Noto Sans in KDE became worse after this PR. Is there any chance to add ability to use non-variable weight variant?

[jopejoe1]

Gone look into that this weekend.

[wkral]

As you were waiting for complaints I think I have one with the variable fonts. They don't seem to work with xelatex.

I got the following error, after expanding the steps to get more verbosity:

xdvipdfmx:warning: Invalid TTC index (not TTC font): /nix/store/y95rbj8y3jpzzddsafvdq6i6zf03xh70-noto-fonts-23.5.1/share/fonts/noto/NotoSans[wdth,wght].ttf

There is a breaking changes announcement thread on discourse, which would make this sort of thing easier to find, it took some nixpkgs code spelunking for me to figure out what that error even meant and what was going wrong to eventually figure out a workaround.

For anyone else who finds this, my workaround was to use the Nerd Font version of Noto as that currently hasn't switched to variable fonts.

[Mathnerd314]

Yeah, it does seem from this discussion that XeTex does not support variable fonts, and has basically been undeveloped font-support-wise for several years. There is this overleaf article mentioned. which says "XeTeX cannot load raster-based OpenType colour fonts" but that LuaHBTeX can. Also it seems LuaHBTeX is more actively developed. Furthermore this SO answer says variable fonts should work in LuaTeX. So maybe try LuaTeX?

[emilazy]

I think it would be a good idea to package the non-variable versions even if they're not default for the long tail of software that doesn't support them and might not for a long time. libass is another example with no real alternative. "Just switch LaTeX version", while maybe a good idea for other reasons, is awkward advice to follow for the sake of a single font.

[Mathnerd314]

Well, there is updating, and then there are workarounds for the software that isn't updated. I don't think it's that hard to just download the font manually and point LaTeX to it. I myself mostly use Windows fonts, downloaded and installed into the Nix Store with nix-add or some such voodoo. In contrast libass takes some work to specify dependencies and build options, so its package actually is nontrivial. But I guess there could be a package for the non-variable fonts. I actually don't know if NixOS has a policy on what level of use a package needs before it is worthy of inclusion in the main repository.

[emilazy]

Even though I like variable fonts and am broadly in favour of keeping them as the default, and would personally prefer stricter inclusion standards than nixpkgs currently has, I think "a version of one of the most popular, high-coverage open source fonts that works with a long tail of other software we package, some actively maintained, that doesn't support variable fonts and is unlikely to for some time" is probably more useful than the majority of packages in nixpkgs. Especially since it worked fine before.

For what it's worth, Fedora (which was among the first to default to the variable font versions) still offers non-vf packages with the separate weights.

[wkral]

Thanks for looking into that @Mathnerd314. Indeed LuaTeX seems to work for me as a drop in replacement in this case.

I agree with @emilazy that perhaps there could be a fixed varient of the package for folks using some software that doesn't have an easy substitute like I do. I originally just got an error that was something to the effect of Invalid font without changing anything other than nixpkgs version. I only became aware that variable fonts could be a problem by finding this commit in nixpkgs after checking many other places for errors. So it would be good to have an easy solution for people who encounter similar problems.