Streamlining translation resource loading in React apps with babel-plugin-preval

At Neeto, our product development involves reusing common components, utilities, and initializers across various projects. To maintain a cohesive and standardized codebase, we've created specialized packages, or "nanos" such as neeto-commons-frontend, neeto-fields-nano, and neeto-team-members-nano.

neeto-commons-frontend houses utility functions, components, hooks, configuration settings etc. neeto-fields-nano manages dynamic field components, while neeto-team-members-nano handles team member management functionalities.

These nanos, along with others, reduce redundancy and promote consistency across our products.

Translation Challenges

Many of our packages export components with text that requires internalization, maintaining their own translation files. We encountered an issue with the withT higher-order component (HOC) using react-i18next inside neeto-commons-frontend. Upon investigation, we found discrepancies in how packages handled dependencies.

withT is an HOC which provides the t function from react-i18next to the wrapped component as a prop.

1import { withTranslation } from "react-i18next";
2
3const withT = (Component, options, namespace = undefined) =>
4  withTranslation(namespace, options)(Component);
5
6export default withT;
Copy

1// Example usage of withT:
2const ComponentWithTranslation = withT(({ t }) => <div>{t("some.key")}</div>);
Copy

Let us first understand the difference between dependencies and peerDependencies. dependencies are external packages a library relies on, automatically installed with the library. peerDependencies suggest that users should explicitly install these dependencies in their application if they want to use this library. If not installed, we will get warnings during installation of this library to prompt us to install the peer dependencies.

react-i18next and i18next were listed as peerDependencies in neeto-commons-frontend's package.json. So, it will be using the instances of these libraries from the host application.

Examining neeto-fields-nano, we found that it listed react-i18next and i18next as dependencies rather than peerDependencies. This meant it had its own instances of these libraries, leading to initialization discrepancies.

Contrastingly, neeto-team-members-frontend listed react-i18next and i18next as peerDependencies, relying on the host application's initialization of these libraries.

The initialization logic, which is common among all the products is placed inside neeto-commons-frontend. To ensure translations from all packages, including neeto-commons-frontend are merged with that of the host application, we crafted a custom initializeI18n function:

1import DOMPurify from "dompurify";
2import i18n from "i18next";
3import { curry, mergeAll, mergeDeepLeft } from "ramda";
4import { initReactI18next } from "react-i18next";
5
6import commonsEn from "../translations/en.json";
7
8const packageNames = [
9  "neeto-molecules",
10  "neeto-integrations-frontend",
11  "neeto-team-members-frontend",
12  "neeto-tags-frontend",
13];
14
15const getPackageTranslations = (language, packageNames) => {
16  const loadTranslations = curry((language, packageName) => {
17    try {
18      return require(`../${packageName}/src/translations/${language}.json`);
19    } catch {
20      return {};
21    }
22  });
23  const allTranslations = packageNames.map(loadTranslations(language));
24
25  return mergeAll(allTranslations);
26};
27
28const packageEnTranslations = getPackageTranslations("en", packageNames);
29
30const en = mergeDeepLeft(commonsEn, packageEnTranslations);
31
32const initializeI18n = resources => {
33  i18n.use(initReactI18next).init({
34    resources: mergeDeepLeft(resources, { en: { translation: en } }),
35    lng: "en",
36    fallbackLng: "en",
37    interpolation: { escapeValue: false, skipOnVariables: false },
38  });
39};
40
41export default initializeI18n;
Copy

Here we are looping through all the packages mentioned in packageNames and merging with the translation keys inside neeto-commons-frontend, along with the translation keys from the host app passed as an argument to initializeI18n function.

While this approach successfully merges translations, it introduced complexity. As our project expanded with the inclusion of more packages, we found the need to regularly update the neeto-commons-frontend code, manually adding new packages to the packageNames array. This prompted us to seek an automated solution to streamline this process.

Given that all our packages are under the @bigbinary namespace in npm, we explored the possibility of dynamically handling this. An initial thought was to iterate through packages under node_modules/@bigbinary and merge their translation keys. However, executing this in the browser was not possible since the browser does not have access to it's build environment's file system.

Enter babel-plugin-preval:

To automate our translation aggregation process, we turned to babel-plugin-preval. This plugin allows us to execute dynamic tasks during build time.

babel-plugin-preval allows us to specify some code that runs in Node and whatever we module.exports in there will be swapped.

Let us look at an example:

1const x = preval`module.exports = 1`;
Copy

will be transpiled to:

1const x = 1;
Copy

With preval.require, the following code:

1const fileLastModifiedDate = preval.require("./get-last-modified-date");
Copy

will be transpiled to:

1const fileLastModifiedDate = "2018-07-05";
Copy

Here is the content of get-last-modified-date.js:

1module.exports = "2018-07-05";
Copy

Here, the 2018-07-05 date is read from the file and replaced in the code.

In order to use this plugin we just need to install it and add preval to the plugins array in .babelrc or .babel.config.js

Streamlining Translations with preval:

We revamped the initializeI18n function using preval.require to dynamically fetch translations from all @bigbinary-namespaced packages. This eliminated the need for manual updates in neeto-commons-frontend whenever a new package was added.

With preval, our initializeI18n function was refactored as follows:

1const initializeI18n = hostTranslations => {
2  // eslint-disable-next-line no-undef
3  const packageTranslations = preval.require(
4    "../configs/scripts/getPkgTranslations.js"
5  );
6
7  const commonsTranslations = { en: { translation: commonsEn } };
8
9  const resources = [
10    hostTranslations,
11    commonsTranslations,
12    packageTranslations,
13  ].reduce(mergeDeepLeft);
14};
Copy

The code for getPackageTranslations.js:

1const fs = require("fs");
2const path = require("path");
3
4const { mergeDeepLeft } = require("ramda");
5
6const packageDir = path.join(__dirname, "../../");
7
8const getPkgTransPath = pkg => {
9  const basePath = path.join(packageDir, pkg);
10
11  const transPath1 = path.join(basePath, "app/javascript/src/translations");
12  const transPath2 = path.join(basePath, "src/translations");
13
14  return fs.existsSync(transPath1) ? transPath1 : transPath2;
15};
16
17const packages = fs.readdirSync(packageDir);
18
19const loadTranslations = translationsDir => {
20  try {
21    const jsonFiles = fs
22      .readdirSync(translationsDir)
23      .filter(file => file.endsWith(".json"))
24      .map(file => path.join(translationsDir, file));
25
26    const translations = {};
27
28    jsonFiles.forEach(jsonFile => {
29      const content = fs.readFileSync(jsonFile, "utf8");
30      const basename = path.basename(jsonFile, ".json");
31
32      translations[basename] = { translation: JSON.parse(content) };
33    });
34
35    return translations;
36  } catch {
37    return {};
38  }
39};
40
41const packageTranslations = packages
42  .map(pkg => loadTranslations(getPkgTransPath(pkg)))
43  .reduce(mergeDeepLeft);
44
45module.exports = packageTranslations;
Copy

In this workflow, we iterate through all the packages to retrieve their translation files and subsequently merge them. We are able to access the translation files of our packages since we have exposed those files in the package.json of all our packages.

files property in package.json is an allowlist of all files that should be included in an npm release.

Inside package.json of our nanos, we have added the translations folder to the files property:

1{
2  // other properties
3  files: ["app/javascript/src/translations"];
4}
Copy

It's worth noting that we won't run preval at the time of bundling neeto-commons-frontend. Our objective is to merge the translation keys of all installed dependencies of the host project with those of the host project itself. Since neeto-commons-frontend is one of the dependencies of the host projects, executing preval within neeto-commons-frontend is not what we needed.

Consequently, we've manually excluded the preval plugin from the Babel configuration specific to neeto-commons-frontend:

1module.exports = function (api) {
2  const config = defaultConfigurations(api);
3  config.plugins = config.plugins.filter(plugin => plugin !== "preval");
4  config.sourceMaps = true;
5};
Copy

With this change, the Babel compiler simply skips the code for preval during build time and the preval related code will be kept as it is after compilation for neeto-commons-frontend.

Another challenge arises from the default behavior of webpack, which does not transpile the node_modules folder by default. However, it's necessary for our host application to perform this transpilation. To address this, we wrote a custom rule for webpack. The webpack rules are also placed in neeto-commons-frontend and shared across all the products.

1  {
2    test: /\.js$/,
3    include:
4      /node_modules\/@bigbinary\/neeto-commons-frontend\/initializers\/i18n/,
5    use: { loader: "babel-loader", options: { plugins: ["preval"] } },
6  },
Copy

This configuration ensures that Babel applies the necessary transformations to the code located in node_modules/@bigbinary/neeto-commons-frontend/initializers/i18n/ within the host application.

Upon transpilation, our system consolidates all translations from each package, including those from the neeto-commons-frontend package, and incorporates them into the host application.

To mitigate potential conflicts arising from overlapping keys, we've implemented a namespacing strategy for translations originating from various packages. This ensures that translations from our packages carry a distinctive key, uniquely identifying their source.

As an illustration, consider the neeto-filters-nano package. In its English translation file (en.json), the translations are organized within a dedicated namespace:

1neetoFilters: {
2    "common": { }
3}
Copy

Conclusion:

Leveraging babel-plugin-preval significantly simplified our translation resource loading process. The automation introduced not only streamlined our workflow but also ensured that our applications stay consistent and easily adaptable to future package additions.