Set user language from HTTP Accept-Language by Beuc · Pull Request #8751 · emscripten-core/emscripten

Beuc · 2019-06-06T10:49:52Z

This allows users to start the app/game with LANG set to their preferred language, avoiding the need to explicitly change it in the app's GUI, POSIX-style.
e.g.

Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
> navigator.languages = Array(4) [ "fr", "fr-FR", "en-US", "en" ]
> LANG=fr.UTF-8

kripken · 2019-06-07T00:40:42Z

      ENV['LANG'] = 'C.UTF-8';
+      if (navigator.languages != 'undefined' && navigator.languages.length > 0) {
+          ENV['LANG'] = navigator.languages[0].replace('-', '_') + '.UTF-8';
+      }


This change seems a little risky to me - maybe some language has - in other places? Also, the app may behave differently for different users, which is the point I guess :) but it may be surprising to start doing so now.

To improve Closure minifiability, one could do:

ENV['LANG'] = (typeof navigator === 'object' && navigator.languages && navigator.languages[0] || 'C').replace('-', '_') + '.UTF-8';

(I would assume that if navigator.languages exists, it is an array)

Beuc · 2019-06-07T09:17:00Z

the app may behave differently for different users, which is the point I guess :) but it may be surprising to start doing so now.

Here, I wish to pass the language information from the browser to the application, which is currently lacking, so that i18n can be initiated.
I added a note in ChangeLog.md to describe the new behavior.
I understand this may cause the application's behavior to differ (note: it needs to call setlocale() to experience any effect), though I believe this new behavior is better for portability.
I also appreciate that this is a low-level change, and I'm open to other implementation ideas. Do you want me to start a thread on the mailing list?

This change seems a little risky to me - maybe some language has '-' in other places?

According to the list of languages in Firefox > Preferences > Languages, no (only xxx or xxx-XXX codes).
According to RFC7231 pseudo-languages like x-pig-latin could theoretically happen, in which case this will result in proposing invalid locale x_pig-latin, which a traditional C program would just ignore.
MDN also references spelling variants like de-DE-1996 which are "usually not used in the context of this header" and would result in de_DE-1996 which I believe is correct.

kripken · 2019-06-11T00:58:39Z

Interesting, thanks.

Overall I feel I don't know enough about this to say what's right. I especially worry since the MDN page on this feature says it is experimental. So, yeah, maybe asking on the mailing list is a good idea. If there are no strong feelings against it then this seems ok to me, the risk is probably low enough.

Btw, why would this change only be noticeable by calling setlocale? It seems like anything that looks at that environment value could be affected? (which I agree is probably very few things though)

kripken · 2019-06-11T00:59:11Z

      ENV['HOME'] = '/home/web_user';
      ENV['LANG'] = 'C.UTF-8';
+      if (typeof navigator === 'object' && typeof navigator.languages === 'object' && navigator.languages.length > 0) {
+          ENV['LANG'] = navigator.languages[0].replace('-', '_') + '.UTF-8';


(should be 2-space indentation)

Beuc · 2019-06-11T19:29:46Z

the MDN page on this feature says it is experimental

navigator.language (without 's') is old/"standard" but may still return the browser's internal language pack, rather than the requested webpage languages from the user's preferences.
navigator.languageS, which this patch relies on, correctly returns HTTP Accept-Language, is more recent and is supported by all browsers that support WebAssembly AFAICS. For older browsers, no changes.

Btw, why would this change only be noticeable by calling setlocale?

I meant that standard C library functions like scanf(3) are affected only when calling setlocale(3).
If the program directly checks the environment variable then indeed there's a change :)

Beuc · 2019-06-11T19:59:31Z

There we go: https://groups.google.com/forum/#!msg/emscripten-discuss/UkJSLsBF3BY/in0QxNQ6BwAJ

Beuc · 2019-06-16T09:10:05Z

Following the discussion on the mailing list:

nobody complained so I believe people are generally fine with this PR
I gave 2 direct use cases with gettext and python.locale

Surprisingly, nobody suggested we did something for nodejs, where I believe it'd make more sense to inherit LANG from the system environment - but since all environment variables are reset that sorta makes sense too.

What do we do with this PR ? :)

juj · 2019-06-16T12:48:11Z

This PR looks good to me - it does bridge POSIX language feature with browser API in a straightforward expected way, and developers do have a way to enforce no locale from both C (setlocale(...)) and JS (ENV['LANG'] = ....

kripken · 2019-06-17T23:48:15Z

I agree, after the discussion I think this is a good approach.

Please just add a test before we land.

…ility (by juj)

Beuc · 2019-06-23T13:44:07Z

I got some time at last and implemented the test case!
(I mocked navigator.languages, as I didn't see any way to set custom browser headers and/or reconfigure the browser settings in the current test harness.)

kripken · 2019-06-24T16:19:12Z

      ENV['HOME'] = '/home/web_user';
      ENV['LANG'] = 'C.UTF-8';
+      // Browser language detection #8751
+      ENV['LANG'] = (typeof navigator === 'object' && navigator.languages && navigator.languages[0] || 'C').replace('-', '_') + '.UTF-8';


How does the order of evaluation work here - is && before || or the opposite? Might be clearer to add parens, I think.

kripken · 2019-06-24T16:22:05Z

+  // Expected results:
+  // LANG=C: 1 / 3.4
+  // LANG=fr_FR.UTF-8: 1,2 / 3
+  // Note: locale needs to be installed (cf. `locale -a`)


I'm not sure what this means - it needs to be installed locally? What error happens if it isn't?

kripken · 2019-06-24T16:23:42Z

Thanks, looks good aside from those minor comments! CI errors look unrelated too.

Beuc · 2019-06-24T18:12:57Z

I added clarifications.

kripken · 2019-06-25T20:05:48Z

Great, thanks @Beuc!

This allows users to start the app/game with LANG set to their preferred language, avoiding the need to explicitly change it in the app's GUI, POSIX-style. e.g. Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3 > navigator.languages = Array(4) [ "fr", "fr-FR", "en-US", "en" ] > LANG=fr.UTF-8

kripken reviewed Jun 7, 2019

View reviewed changes

kripken reviewed Jun 11, 2019

View reviewed changes

Beuc force-pushed the patch-5 branch from 42a440b to d21786b Compare June 11, 2019 19:12

Beuc added 3 commits June 23, 2019 15:27

Set user language from HTTP Accept-Language

d29747a

Set user language from HTTP Accept-Language: test case

eef99c9

Set user language from HTTP Accept-Language: improve Closure minifiab…

4820689

…ility (by juj)

Beuc force-pushed the patch-5 branch from d21786b to 4820689 Compare June 23, 2019 13:38

kripken reviewed Jun 24, 2019

View reviewed changes

Set user language from HTTP Accept-Language: clarifications

f925009

kripken merged commit 0ffcb05 into emscripten-core:incoming Jun 25, 2019

Beuc mentioned this pull request Aug 19, 2019

Fix duplicate assignment #9258

Merged

Beuc deleted the patch-5 branch November 1, 2019 15:44

ilonatommy mentioned this pull request Feb 22, 2023

Do not overwrite LANG on NodeJS #18816

Closed

complexspaces mentioned this pull request Apr 2, 2023

Fix support for WebAssembly 1Password/sys-locale#18

Merged

Conversation

Beuc commented Jun 6, 2019

Uh oh!

kripken Jun 7, 2019

Choose a reason for hiding this comment

Uh oh!

juj Jun 16, 2019

Choose a reason for hiding this comment

Uh oh!

Beuc commented Jun 7, 2019

Uh oh!

kripken commented Jun 11, 2019

Uh oh!

kripken Jun 11, 2019

Choose a reason for hiding this comment

Uh oh!

Beuc commented Jun 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Beuc commented Jun 11, 2019

Uh oh!

Beuc commented Jun 16, 2019

Uh oh!

juj commented Jun 16, 2019

Uh oh!

kripken commented Jun 17, 2019

Uh oh!

Beuc commented Jun 23, 2019

Uh oh!

kripken Jun 24, 2019

Choose a reason for hiding this comment

Uh oh!

kripken Jun 24, 2019

Choose a reason for hiding this comment

Uh oh!

kripken commented Jun 24, 2019

Uh oh!

Beuc commented Jun 24, 2019

Uh oh!

kripken commented Jun 25, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Beuc commented Jun 11, 2019 •

edited

Loading