[pre wip] ways to translate help patches.

[Lucarda]

In #3 I'm proposing:

Since the help patches can be translated it could be nice if we find a way to distribute the "translated package" via Deken. Then we can have a way to tell Pd to go and fetch help files in that folder.

Further details on how to do this can be discussed in this thread.

[umlaeute]

translatable help-patches are probably easy to implement (e.g. check for metro-help.pt.pd (if the current locale is pt), and fall back to metro-help.pd if that is missing.

the idea with deken-installable language extension packs is nice (as it's virtually impossible that we will be able to merge translated help-patches into Pd's main branch in time for a new release), but i'm not sure whether it would work out of the box.
deken won't install to the default help-path (and i think we don't really want to change that).

• we might instruct users to add the newly created path to the help-paths. but
  1. i don't know whether this (help-path trickery) actually would works
  2. iirc there's no user-interface in Pd's preferences that allows to set the help-path (which makes it hard for non-console-savy users to perform correctly)
  3. up until now i considered the -helppath flag totally useless and would have loved to see it go away
• we might extend deken to allow for installations of special packages (e.g. references, GUI-translations,...) into "other" places (besides the externals search-path) (though i'm afraid that this will become rather cumbersome quickly)
• in addition, we might extend the user-specific paths to include places for these special packages (e.g. ~/local/lib/pd/help, ~/Documents/Pd/help/), but I think that doesn't work so well with the none-Documents default paths on macOS on Windows.

[Lucarda]

May be we can have a "translations" folder created at the same time ~/Documents/Pd is created (on first run).

Then we use ~/Documents/Pd/translations as the only folder translations go and are searched for.

We can have a new special deken arg "translation" so that the package gets extracted in that folder. e.g
español-5.referencia-(translation).dek

[umlaeute]

the problem with ~/Documents/Pd is that it is

• purely optional, and can be discarded at the users discretion (by just selecting "No" when starting Pd for the first time)
• a GUI-only hack

most people will probably select "Yes" anyhow, but I think a proper solution should be robust against such casualities and provide a uniform way for whatever search path people chose to use.

---

the problem with "a new special deken arg" is that - as proposed - it is an adhoc-solution that does not look very scalable.

• we need something more robust and extensible
• i think "translation" is the wrong tag. afaict we are talking about a documentation overlay, that happens to be in an alternative language. other documentation overlays might include updated reference files in English. the fun part starts when it comes to other overlays - outside the realm of documentation.
• in any case the proposed scheme collides with the current deken-1 filename format (where it would be simply parsed as a library español-5.referencia- (with a trailing dash. wtf) and an architecture of translation. 🤦 )

[Lucarda]

Cool. So we can have a deken-2 filename format file: español-5.referencia.wtf that gets extracted to:

macOS:

~/Library/Pd

Windows:

%AppData%\Pd

GNU/Linux:

~/.local/lib/pd/extra

One thing that I was thinking is to not have the possibility to have more than one "translation package" installed. But may be I'm wrong and there's an easy and robust way of allowing them. Still i think there's no need for that.

[umlaeute]

Am 6. Juni 2021 00:44:14 MESZ schrieb Lucas Cordiviola ***@***.***>:

Cool. So we can have a **deken-2** filename format file: `español-5.referencia.wtf`

that doesn't look like a robust, parsable filename.

that gets extracted to: macOS:> > ` ~/Library/Pd` > > Windows:> > ` %AppData%\Pd `> > GNU/Linux:> > ` ~/.local/lib/pd/extra` >

How's that different from what we already have?

> One thing that I was thinking is to _not_ have the possibility to have more than one "translation package" installed. But may be I'm wrong and there's an easy and robust way of allowing them.

that actually looks like the simplest problem here (if we allow for changes in the pd-core). Just check how other translation systems handle this.

[Lucarda]

that doesn't look like a robust, parsable filename.

I'm thinking that we don't need much parsing for translations. Also I thought that a new extension would act as a first router.

[Español](doc/5.referencia)@2021-06-06.tra

May be the date part is redundant as this can be taken from upload time.

But may be you have other plans for deken-2 filename format. I guess your plans does not include a new file extension and .dek files have to evolve for handling externals, translations and more.

at least in my head is simpler to have (in the server) a new file extension for translations.

May be this can later be discussed as an implementation detail in Deken's repo.

---

~/.local/lib/pd/extra
How's that different from what we already have?

Well is not different but the user specific directory we already have does not depend on decisions from the user.
Please elaborate why we should not use user specific.

---

One thing that I was thinking is to not have the possibility to have
more than one "translation package" installed. But may be I'm wrong and
there's an easy and robust way of allowing them.

that actually looks like the simplest problem here (if we allow for changes in the pd-core).
Just check how other translation systems handle this.

I'm glad this is simple and doable. might be useful for translators.

[Lucarda]

TODO:

Define a folder structure for Translations.

[Lucarda]

Maybe the deken file name could be something:

[<language name>](<description>){<extraction dir>}.trs

[Español]("Ayuda de objetos internos"){ES/doc/5.reference}.trs

[umlaeute]

better (as it's parsable!).

however, i still would like to use a general deken filename format for that, that also covers the general use-case for deken (namely: installing libraries).
in any case

• we need versions!
• the filename needs to map to valid URLs
• the filename must not exceed any filename length limits (for any modern filesystem except 8.3-like fs)
• the filename must be easily creatable by humans and scripts
• we don't want a generic description in the filename
• there is no use in encoding the same information twice ("ES" vs "Español")
  • what do you make of a filename [Español]("Помош за внатрешен објект"){KO/doc/5.reference}.trs?

[umlaeute]

Define a folder structure for Translations.

without much thinking I see two possible solutions:

• use language specific files

  /path/to/Pd/doc/5.reference/bang-help.pd
  /path/to/Pd/doc/5.reference/bang-help.de.pd
  ~/Library/Pd/doc/5.reference/bang-help.pt_br.pd
  

• use language specific folders

  /path/to/Pd/doc/5.reference/bang-help.pd
  /path/to/Pd/doc/5.reference/de/bang-help.pd
  ~/Library/Pd/doc/5.reference/pt_br/bang-help.pd
  

• (there's also the possibility to have a single folder that contains all the translations for a given language, e.g. %AppData%\Pd\pt_br\doc\5.reference\bang-help.pd. i dislike this as it groups data by something rather arbitrary (the language) instead of functionality ("reference patches"))

intuitively i prefer a filename-suffix that indicates the language (bang-help.de.pd), but intuition can be deceptive.

• concise folder-structure
• it requires explicit support for language-suffixes in Pd
• it can get quite messy if you have multiple languages installed.
• might be easier to integrate in the current loading mechanism (i haven't checked)

the directory-suffix could be turned into a more generic solution

• bloated folder-structure
• makes it somewhat harder to compare multiple languages
• rather than explicit support for language-directories, we could add generic support for dynamically adding (persistent) help-paths to Pd (similar to what we already have for search-paths)

[umlaeute]

another rock on the road: what about versioning?

1. the help-patch for [bang] gets translated into portuguese of Pd-0.51
2. [bang] gets super-cow powers for Pd-0.52
3. the master-help patch for [bang] gets updated to document the new powers
4. the user installs Pd-0.52 and the portuguese language pack (for Pd-0.51)

which help-patch do they see?

[Lucarda]

about the filename

ES[v0.51.4]{doc/5.reference}.dek

<locale>[<pd version>]{<extraction dir>}

about folder structure.

At least on normal Pd Windows installations we can discard /path/to/Pd/doc/5.reference/bang-help.de.pd. %programfiles% directories are not writable for Pd or Deken (ie :permission denied warning).

I think the user specific folder is the way to go.

<user specific>/Pd/<dek.locale>/<dek.extraction dir> give us the functionality we need.

i dislike this as it groups data by something rather arbitrary (the language) instead of functionality ("reference patches"))

You can also like it because it groups the "translations" :)

the directory-suffix could be turned into a more generic solution

• bloated folder-structure

"more generic solution" looks good to me. It might be bloated but not at all useless (more the contrary). (also note the above mention to the unwritable Windows thing)

• makes it somewhat harder to compare multiple languages

really?

rather than explicit support for language-directories, we could add generic support for dynamically adding (persistent) help-paths to Pd (similar to what we already have for search-paths).

Isn't this going to break all the "Deken auto install dir". I'm not saying no, just want to make sure if there's a need for that. The way I see it "dynamically adding (persistent) help-paths to Pd" is what is going to be done to the "Deken auto install dir".

another rock on the road: what about versioning?

which help-patch do they see?

I think we must live with it. No clues on what to do about this.

[danomatika]

which help-patch do they see?

I think we must live with it. No clues on what to do about this.

If the folder structure includes the Pd version or there is a meta file, it would be possible to compare...

[Lucarda]

Should be more fine grained with a per file [meta].

We can mention this on #6 so that a [meta] gets into the new [pd reference] sub-patch

EDIT:

This can be less hidden from the translator and just go in [pd reference] (instead of [pd reference]/[meta)]

Edit 2: I think i got totally confused about [meta] and don't like the idea of having that much of *-meta.pd laying around.

[umlaeute]

about the filename

ES[v0.51.4]{doc/5.reference}.dek

<locale>[<pd version>]{<extraction dir>}

the filename is always something that is presented to the humans in one way or the other.
e.g. when searching for "zexy", the first line i see says zexy[v2.3.1](Darwin-amd64-32)(Linux-amd64-32)(Linux-arm64-32)(Linux-armv7-32)(Linux-i386-32)(Windows-amd64-32)(Windows-i386-32).dek. there's a lot of garbage in this (and i deliberately chose this as an example) that I - as a human - quickly filter out (practically everything after the first (Dar.... but still I see "zexy" in there (which is reassuring, as this was the term in searched for). i also see this "[v2.3.1]" string, which doesn't mean much by itself. but I also see other search results that contain similar strings (like "[v2.3.10]" and "[v2.2.8]"), and those strings are sorted in descending order. so probably it is a version.
compared to that i think that ES[v0.51.4]{doc/5.reference}.dek is too cryptic.

but to be more specific:

• i don't know why you want the locale to be included in the filename. the locale will be present somehow in the structure of the extracted files (either in the extracted file names or in the extracted directory names).
• the Pd-version is the Pd-version; but that does not necessarily match the translation version (e.g. if you have multiple revisions of the translations)
  similarily, most parts of the extraction dir can be specified by the contents of the zip-archive.
  but not all of them. e.g. currently the unarchive-process expects to put all files into some "extra/"-directory (it might not actually contain the name "extra"; but conceptually it does), and not into doc/ or whatever.
  we really don't want to encode the same information twice (e.g. specifying doc/5.reference as both the {<extraction_dir>} and as the filename in the archive doc/5.reference/bang-held.pd). and we don't really want to allow to encode the same information in multiple ways (e.g. specifying doc as the {<extraction_dir>} and the filename in the archive is 5.reference/bang-held.pd; resp. doc/5.reference as the {<extraction_dir>} and the filename in the archive being bang-held.pd).

the more powerful such tags get, the more dangerous they become. we really must protect the users from a package

also note that the / is simply a forbidden character for filenames in practically all filesystems (including those found on W32 machines)

about folder structure.

At least on normal Pd Windows installations we can discard /path/to/Pd/doc/5.reference/bang-help.de.pd. %programfiles% directories are not writable for Pd or Deken (ie :permission denied warning).

that's totally beside the point.
i expect a similar mechanism to work as with current deken - and deken does try to install to %ProgramFiles%/Pd/extra: it just detects whether this path is writable and skips it if not.

i also expect whatever solution we come up with to hook into dekens installation folder mechanism: if I - as a user - chose to install into /tmp/dudeldum/ that i would be seriously annoyed if the translations were installed into ~/Documents/Pd/.

I think the user specific folder is the way to go.

<user specific>/Pd/<dek.locale>/<dek.extraction dir> give us the functionality we need.

i dislike this as it groups data by something rather arbitrary (the language) instead of functionality ("reference patches"))

You can also like it because it groups the "translations" :)

yes. but that is a very bad grouping.

we could also group files based on their first character (b/doc/5.reference/bang-help.pd, c/doc/5.reference/change-help.pd), but that doesn't make much sense.
we are currently talking about i18n, so different languages are "natural" categories to group our thoughts.
but normally (when working with Pd) this is not the case. we group our thoughts by workflows.
e.g.

• "i'm bored. let's check what some random translator provided me with in terms of documentation"
• "i'm bored let's browse the documentation. hopefully i'll find some in my native language"

it also does not touch the question of multilanguage help-patches for libraries.
(or do you think we should install ELSE into the pt_br folder, pulqi into the es folder, and zexy into de_at?)

rather than explicit support for language-directories, we could add generic support for dynamically adding (persistent) help-paths to Pd (similar to what we already have for search-paths).

Isn't this going to break all the "Deken auto install dir". I'm not saying no, just want to make sure if there's a need for that. The way I see it "dynamically adding (persistent) help-paths to Pd" is what is going to be done to the "Deken auto install dir".

i'm not sure i understand your concerns.

[Lucarda]

I got scared of getting the value directly from the registry in C:

https://stackoverflow.com/questions/34065/how-to-read-a-value-from-the-windows-registry

[Lucarda]

it's only supported on >=Vista (i've no idea, whether Pd still runs on XP out of the box. But i think it is still compilable)

Yes Pd is still working on XP but not ipv6 networking. May be also without translations? :)

[Lucarda]

this c code gets values from the registry. is working for me on spanish and english.

RegQueryValueEx.zip

[umlaeute]

thanks.
in the meantime, i came up with my own implementation (it took so long as i did not spot a simple LocalName typo...)

check out the latest and greatest feature/i18n-help branch (currently at d7d713) and try again.
(this also uses LocaleName on the GUI, if it is available; so check out if the ordinary GUI l10n is broken as well ;-))

[Lucarda]

Ok the translated patch now works but the rest of Pd is no longer translated as before. (on the spanish user) (the english user just works as always)

I have to stop and sleep.

:_)

[umlaeute]

should be fixed now.

PS: i was already wondering about your sleep-cycles.

[porres]

those of us in Argentina and Brazil have weird sleep cycles :)

[Lucarda]

should be fixed now.

Yeah. Tested and working fine. :)

[umlaeute]

so I got a working protoype.
it requires:

i've added some snapshot binaries to the description of pure-data/pure-data#1340
with these you can always test the latest and greatest feats.
(as a sidneote: the macOS binaries should be signed and notarized; i would also be grateful for feedback whether this works (though probably not here))

[Lucarda]

[feature request]

It would be useful if we have a way, standing in a translated patch, to open the non-translated patch.

Something with a GOP as shown above. The GOP could have something like

[openstatic abs~-help.pd(
|
[s pd]

I'm only thinking this for the "5.reference" patches as this have more priority for the users and translators.

[umlaeute]

good idea.

i was working on a [file] object (family) that comes with a [file which] object that does file-lookup.

with that, you could do something like this:

[symbol abs~(
|
[symbol ../doc/5.reference/$1-help.pd(
|
[file which]
|
[; pd open $1 /(

the plan is, to add a PR to include file into Pd.

PS: this obviously relies on not having a ../doc/5.reference/abs~-help.pd file anywhere in the path (except for the original helppatch, that lives in /path/to/pd/extra/../doc/5.reference/)
you could add an additional safeguard and use ../extra/../doc/5.reference/$1-help.pd

[umlaeute]

seems like the [file] object will be part of Pd-0.52 (pure-data/pure-data@7ef5bc8)

[abs~(
|
[symbol]
|
[symbol ../extra/..doc/5.reference/$1-help.pd(
|
[file which]
|
[file splitname]
|
[; pd open $1 $2(

i also think this could mean that we can get away without having any brittle magic to detect whether the translated help-patch is more recent than the original one (or the other way around)

[Lucarda]

seems like the [file] object will be part of Pd-0.52

:)

we can get away without having any brittle magic to detect whether the translated help-patch is more recent than the original one (or the other way around)

Agree.

[porres]

I see I never really participated on this, besides making a joke about bad sleeping habits.

I like the idea of having translated help files, the question is, it doesn't feel as a sane workload. Over the years I have been constantly rewriting and revising help files, and to do that and update several translation options would have been a disaster I guess... it wouldn't keep up.

I guess the tendency is to stabilize over time and have less edits, but it's also true that at least new features pop up every release and a good amount of change is needed.

I just don't see much people doing this. Anyone could revise and rewrite help files, but I ended up doing basically all that. As for translating to portuguese-BR, I don't see anyone up for it other than me, and I know I'm not up for it.

So, yeah, a nice idea, but not much practically feasible. If we had a bigger and more dedicated community eager to collaborate, maybe...

By the way, I think it's more crucial and important to translate Pd strings itself with weblate and stuff, and I don't see that as fully accomplished as it could be yet.

Anyway, it's been 3 years, and I would like to clean up a few things and get things that didn't take off out of the way. Is there any desire to keep this going? Does anyone feels different that this would be in fact tangible?

[porres]

hmmm, I also see now this has been split into a discussion and an issue at #3 and that there is also a related PR for Pd Vanilla itself for this at pure-data/pure-data#1340 so there's a lot os scattered information and is hard to follow. Also, the PR in Vanilla has conflicts and is sitting there for a few years. Can we at least close this discussion and reopen it or reboot if the needed PR gets worked on and merged?