Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Planning issue for translation/internationalization/localization systems #3262

Open
jywarren opened this issue Aug 27, 2018 · 87 comments
Open

Planning issue for translation/internationalization/localization systems #3262

jywarren opened this issue Aug 27, 2018 · 87 comments
Assignees
@grvsachdeva
Labels
break-me-up break up for cleaner code separation, discrete tests, and, easier and iterative collaboration outreach issues involve community involvement and helping people who're stuck somewhere planning Planning issues! Translation System issues which involve adding new translations, improving the translation system

Comments

@jywarren
Copy link
Member

jywarren commented Aug 27, 2018 •
edited
Loading

We have some basic internationalization/localization via the i18n gem, but need to think through how to improve it, and ALSO come up with a streamlined workflow to recruit and integrate new translations on an ongoing basis. This is a planning issue for this set of projects.

Translation systems

Our translation files are spread over lots of files and folders in https://github.com/publiclab/plots2/tree/master/config/locales

I'm inclined to make them all one big file, or at least consolidate them some. One folder with key/value would be SO AWESOME! I wasn't sure why our implementation fragmented everything across so many folders. I think we could do this in Transifex.

Discussion of alternative platforms via @ebarry :

Ongoing translation workflow

We need a workflow to regularly go from Transifex (or something similar) back into the translation files.

Transifex has GitHub integration: https://docs.transifex.com/integrations/github/ and awesome ability to auto-sync in TravisCI: https://docs.transifex.com/integrations/github/#integrating-the-client-with-travis-ci

This would grab new translations each time we accepted a pull request, and register new texts that need translation!

Outreach

For actually doing the translations, I'm hesitant to make a huge # of translation issues in the same repository, even if they are first-timer type issues. I think we could maybe get more contributors to translate outside the GitHub interface (which is most welcoming to coders).

I think we'll need a good outreach game to actually recruit people to do translations! Maybe, we could tweet them out? We could show a random phrase at the bottom of the site each time the site is loaded, that needs translation?

If we happen to find alternatives (especially open source ones) that's great, but we can stick with Transifex if there's no big reason to switch.

The ideal might be that with fewer files, we could copy in new changes from either another service like Transifex periodically (monthly?), or from files in another repository (although this doesn't make it more accessible by going outside the GitHub interface)

We might even try to automate the import and run it periodically. But it could be a regular task manually too.

Translation prompts

Anywhere we display translated text using the t('some_string_here') helper, we ought to display a little globe icon with a tooltip saying Needs translation? Click to help translate this text. or something (ideally in the browser-configured language!)

Translating content

We'll need to think about how to translate both regular site content (wiki pages + notes) and ALSO features, which are used around the site for things like the static footer, explanatory text, etc.

Perhaps there should be a language type on revisions, and we could make it possible to track translated revisions alongside regular ones. Or perhaps a translated node, linked to the main node?

Compare https://publiclab.org/barnraising to https://publiclab.org/wiki/barnraising-guangzhou-china for an example of 2 versions of a page in 2 languages.

Features

Some parts of site content -- like the footer and front page -- are run in nodes, as database content. Let's think of a way to have features like frontpage-en vs frontpage-zh and frontpage-es

Helpers for translation

@siaw23 points out, and I agree, that translation is tough for newcomers submitting new code. The current format is something like:

<%= t('some_string_here') %>

...where some_string_here is the key to some string of text.

I propose a new helper, maybe called translate(), which looks for translations, but if none are present, displays the key text itself. That way, we can create new interfaces that include text, without first having to do translation for them -- they'll have the fallback text. So:

<%= translate('Click here to begin') %>

which initially would have only the default english, but via the prompts (above) would encourage community translation efforts.

Scanning for translation opportunities

Finally, we might have a script which scans for uses of the new helper (from above) that don't have entries in the default translation file, and copies them in.

This is a lot. Just kicking off some discussion and we can start to move things into separate issues soon! Love to hear peoples' input!
@jywarren jywarren added the break-me-up break up for cleaner code separation, discrete tests, and, easier and iterative collaboration label Aug 27, 2018
@jywarren jywarren mentioned this issue Sep 5, 2018
Closed
78 tasks
@ebarry ebarry added the outreach issues involve community involvement and helping people who're stuck somewhere label Sep 12, 2018
This was referenced Sep 22, 2018
Closed
Merged
Closed
@jywarren
Copy link
Member Author

I've broken out the "move translations" issue into a multiple-use issue good for newcomers! #3515

@jywarren jywarren mentioned this issue Oct 20, 2018
Closed
@jywarren jywarren mentioned this issue Nov 14, 2018
Closed
@jywarren jywarren added the planning Planning issues! label Jan 2, 2019
@grvsachdeva
Copy link
Member

grvsachdeva commented Jan 8, 2019 •
edited
Loading

Hi @jywarren @ebarry , I went through the details of the project and done some research at my end.

I saw that Transifex is being considered right now for integration, but that was paid. There are two more methods which I am considering:
1. Using Copycopter gem
In this, the web interface would be hosted as a separate application and can be hosted via Heroku too as it only requires one dyno to run.

Although, the project is Abandoned some time back but, it can do our work. For details, check here - https://github.com/copycopter/copycopter-server

2. i18n backend
It is one of the solutions which offers control of the system in our hand. Please check details here- http://railscasts.com/episodes/256-i18n-backends?autoplay=true

I prefer the second option as with that we wouldn't have to depend on other services and also can use the interface of our preference. Let me know if you like any of the above methods or Transifex is final choice.

Thanks!

@jywarren
Copy link
Member Author

jywarren commented Jan 8, 2019 via email

@grvsachdeva
Copy link
Member

@jywarren

In the second option, what backend would we use? Not flat yml files? How
would people edit them then?

In this option, translations can be stored in Redis as a key-value pair. We can use yml files just to provide fallback translation, so we can only have en.yml and others stored in DB. People would be interacting with DB only, just like editing/adding a comment.

Is it possible to use Transifex or another management system to organize
the crowdsourcing of translations, so we could kind of quickly get people
plugged int translating short strings?

I think, Transifex provides translation services too, but again that services would be paid on top of paid Transifex subscription.

I highly recommed watching this http://railscasts.com/episodes/256-i18n-backends?autoplay=true episode for clarity of second method. It's related to our need.

Thanks!

@grvsachdeva grvsachdeva added Translation System issues which involve adding new translations, improving the translation system and removed Translation System issues which involve adding new translations, improving the translation system labels Jan 9, 2019
@grvsachdeva grvsachdeva self-assigned this Jan 9, 2019
@grvsachdeva
Copy link
Member

@jywarren @ebarry I am recording the Translation Project progress at https://github.com/publiclab/plots2/projects/9 . Thanks!

@grvsachdeva grvsachdeva mentioned this issue Jan 9, 2019
Merged
5 tasks
@grvsachdeva
Copy link
Member

@jywarren @ebarry can I create a test project on transifex with publiclab's URL for testing? Thanks!

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

Hi, Gaurav - thanks - i watched the video and perhaps we should open a new issue for this discussion if it gets much longer?

In any case, the feature of Transifex is not the auto-translation, but the idea that people might show up and want to help, but we might want to have translations reviewed, moderated, and managed. The language in this post is a bit overly commercial for us, but some of the principles are the same:

https://www.transifex.com/blog/2016/crowdsourcing-translations-and-moderation-programs/

https://docs.transifex.com/localization-tips-workflows/crowdsourcing-translations has great guidelines for this. The experience for people who offer to translate looks pretty smooth and welcoming, which is nice esp. if many translators aren't coders. And there are different roles: Language Coordinator, Reviewer, and Translator.

What do you think of this? I believe we can do integration as noted above such that the translation texts are under our control and in standard formats.

But the pricing is rough. $140 per month... 😦 @ebarry did you know this? https://www.transifex.com/pricing/

I wonder if there are other options that have some of the desired functions:

  1. easy interface and login for volunteer translators
  2. ability to review translations
  3. maybe ability to have translations "approved" by a second person (reviewer?)
  4. way to use a POST request to link into a specific translation if it doesn't exist, with the text to be translated, so that it will drop the resulting translation into our locale files???

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

And my concern is that Redis and Copycopter don't have the interface or review parts... hmm.

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

Hmm, https://github.com/topics/i18n ?

https://weblate.org/en/hosting/ is based on open source but much cheaper than Transifex

https://github.com/parrot-translate/parrot seems unsupported

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

Ooh this is nice though! https://github.com/glebm/i18n-tasks -- can find unused translation strings and help manage them in a CLI!

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

http://pootle.translatehouse.org/

https://www.accent.reviews/

Hmm interface here is pretty complex: https://demo.weblate.org/translate/hello/master/fr/?type=unapproved&offset=2

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

Whoa.... https://github.com/microsoft/localization/wiki/How-to-contribute-to-Software

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

https://crowdin.com/pricing#annual is $16/month

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

And POEditor has a free tier that might work for us: https://poeditor.com/pricing/

@jywarren
Copy link
Member Author

jywarren commented Jan 9, 2019

I just used localeapp for a while and it's pretty nice. It supports only 2 locales on free tier though. https://www.localeapp.com/pricing

@ebarry can we afford $24/month for 5 locales at some point soon? Or should we try something else. At least this looks easy...

@jywarren
Copy link
Member Author

I think we can go with zh_CN as what I get from google searches is that zh_TW is used majorly in Taiwan and zh_CN is kind of most used/general.

I meant, how does Transifex, in their defaults, address the issue? I think we are OK using primarily zh_CN, but we can revisit if there is interest in supporting both, later. Thank you!

Yes, let's delete and merge! Great!

@grvsachdeva
Copy link
Member

@jywarren transifex added a new method which is awesommmeeee - https://docs.transifex.com/integrations/transifex-github-integration 💯

@grvsachdeva
Copy link
Member

I am deleting old data and then we can do the setup

@grvsachdeva
Copy link
Member

@publiclab/community-reps

I have deleted all the data from transifex.
@jywarren these are languages present now. Good to continue? Any addition or deletion of language?

English, Arabic, Chinese(zh_CN), German, Hindi, Italian, Korean, Portuguese(Brazil)pt_BR, Spanish(es)

@grvsachdeva
Copy link
Member

grvsachdeva commented Mar 15, 2019 •
edited
Loading

@jywarren I sent the Transifex installation request to plots2. I think we can install it now. Also, there are 2 join requests pending on Transifex account -
transifex_req

One is old(2017) and another is by a contributor who told in an issue that they can help in translation so I gave them our account link. So, on which basis we should accept requests?

@jywarren
Copy link
Member Author

jywarren commented Mar 15, 2019 via email

@grvsachdeva
Copy link
Member

Have you approved Transifex app install permissions @jywarren ?

@jywarren
Copy link
Member Author

jywarren commented Mar 18, 2019 via email

@grvsachdeva
Copy link
Member

I tried many times but it's not showing transifex app installed 👇
Screenshot from 2019-03-19 21-30-32

Could you go at https://www.transifex.com/publiclab/settings/integrations/ and install the transifex integration?

Thanks!
@jywarren

@jywarren
Copy link
Member Author

jywarren commented Mar 19, 2019 via email

@jywarren
Copy link
Member Author

jywarren commented Mar 19, 2019 via email

@grvsachdeva
Copy link
Member

ok, I see authorized now
Screenshot from 2019-03-20 00-57-46

@grvsachdeva
Copy link
Member

@jywarren refer this https://docs.transifex.com/integrations/transifex-github-integration

@grvsachdeva
Copy link
Member

they have bot -
Screenshot from 2019-03-20 01-00-09

@jywarren
Copy link
Member Author

jywarren commented Mar 19, 2019 via email

@grvsachdeva
Copy link
Member

I did the remaining setup but the sad part of this integration is we can't make good use of this until we reach 100% reviewed status on a language. I have dropped a suggestion to Transifex regrading this - https://community.transifex.com/t/transifex-should-be-able-to-push-translations-more-often-transifex-github-integration/261

so anyone of us can raise PR weekly till we reach the 100% reviewed status for all langs and after that this integration will raise PR for each reviewed string addition(I will document the steps for PR)?

@jywarren
Copy link
Member Author

jywarren commented Mar 19, 2019 via email

@jywarren
Copy link
Member Author

Hi @gauravano shall we update this and wrap it up? I think we are basically almost there, no? Thank you!!!

@grvsachdeva
Copy link
Member

Hi @jywarren, yes, we are almost done. Give me a day to create a checklist and I guess, I will update the research notes posted by Liz, we can then, start using multiple languages.

Thanks!

@jywarren
Copy link
Member Author

jywarren commented Apr 22, 2019 via email

@grvsachdeva grvsachdeva mentioned this issue Apr 23, 2019
Merged
5 tasks
@jywarren jywarren mentioned this issue Oct 25, 2019
Open
@stale stale bot added the stale label Oct 7, 2020
@jywarren jywarren removed the stale label Oct 8, 2020
@jywarren jywarren mentioned this issue Mar 30, 2021
Closed
@publiclab publiclab deleted a comment from stale bot Mar 30, 2021
@jywarren
Copy link
Member Author

Just noting here that I believe we need an additional "guide" on the process of importing translations from Transifex, perhaps at https://publiclab.org/wiki/translation or https://github.com/publiclab/plots2#internationalization.

@jywarren jywarren mentioned this issue May 11, 2021
Open
@jywarren jywarren mentioned this issue May 25, 2021
Open
@imajit imajit mentioned this issue Jul 16, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@grvsachdeva grvsachdeva

Labels
break-me-up break up for cleaner code separation, discrete tests, and, easier and iterative collaboration outreach issues involve community involvement and helping people who're stuck somewhere planning Planning issues! Translation System issues which involve adding new translations, improving the translation system
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jywarren @ebarry @grvsachdeva