Internationalization

The frontend uses Nuxt’s i18n module to support internationalization, enabling localization of the user interface.

WordPress uses GlotPress for managing translations, which is built on top of the gettext standard. On the other hand, Nuxt (and most JS-based i18n libraries) use JSON for managing translations. This disconnect means that Openverse translations must convert from JSON to POT and back again. Hence, there is quite a bit of scaffolding involved.

Upload pipeline

This pipeline deals with how translations strings are extracted from the frontend application and provided to GlotPress for translation.

Steps

  • Create a POT file from the en.json5 file.

    Script: i18n:generate-pot

  • Upload this POT file to a fixed URL. Currently, the file is hosted in the translations branch of the WordPress/openverse repo.

  • GlotPress presents the strings, fuzzy translations and other helpful context to translators in a web UI to help them provide a translation.

Download pipeline

This pipeline deals with how translations are retrieved from GlotPress, processed and loaded into Nuxt via the Nuxt i18n module. The main entry point, i18n/scripts/setup.mjs, orchestrates the entire process based on the command line arguments. There are three main modes of operation:

  • for production, it can download translations from GlotPress, parse the locales, and update the Vue i18n plugin configuration.

  • for local development, it can convert en.json5 to JSON format used by the Nuxt app, and save the empty valid-locales.json file to be used by the Nuxt app.

  • for testing, it can copy the test locale metadata, and test translations to the main i18n/locales folder.

Production Steps

  • Download all translations from GlotPress as NGX JSON files - flat json files.

    This script downloads all available translations in bulk as a ZIP file and then extracts JSON files from the ZIP file. This prevents excessive calls to GlotPress, which can be throttled and cause some locales to be missed.

    Script: i18n/scripts/translations.mjs

  • Parse and extract the list of all locales from GlotPress’s PHP source code. Then narrow down the list to locales available in the WP GlotPress instance and calculate their coverage percentage from the number of keys in the translation and the number of keys in the main en.json5 file. After that, separate the locale metadata into two groups based on the JSON files emitted by the previous step.

    • translated: JSON file is present with mappings, written to valid-locales.json.

    • (only if –verbose flag is on) untranslated: JSON file is present but empty, written to untranslated-locales.json.

  • Pass the fallback locale mappings to the underlying Vue i18n plugin. This is configured in the plugin configuration file, frontend/src/i18n/vue-i18n.ts.

Test locales

Three locales are maintained in the code base (as opposed to downloaded from GlotPress) for use in testing, alongside the default English locale. These locales are not meant to be representative of actual final translations. They are merely used to approximate length and language features that help identify potential layout issues in the application.

The following languages are used:

  • Arabic (ar)

  • Russian (ru)

  • Spanish (es)

The JSON files for these test locales are located in frontend/test/locales.

In the past, Openverse maintainers updated these locales by hand when new strings were introduced or changed. That is no longer necessary. Instead, use the generate_test_locales utility, which relies on Argos Translate to generate new testing translations on your local machine. It only translates strings for keys that are not already present in the test locales. This means you can freely make manual changes to the generated strings if you feel there is a better way to represent features of a particular string in the test locale languages (for example, if you speak any of these languages and happen to know a generated translation is badly inaccurate).

To use the script:

# Install python dependencies
ov just utilities/generate_test_locales/install

# Run the script
ov just utilities/generate_test_locales/run

The script caches Argos translation packages on your local machine. The first time you run the script, it will take a little longer while it updates the package repository and downloads the necessary translation models. Subsequent runs will start much faster, unless there are updates to the language packages, in which case they will be downloaded.

Due to the nature of how the script traverses the locales, some runs may reorder keys. The order of keys in the JSON file is meaningless, but can create noise in diffs. If it is too noisy for a particular change, you can generate a patch of just the new or changed strings, revert the overall changes, and then apply the patch of new strings (either manually or using git and then resolving conflicts).