Published by Andrew Terris - August 29, 2018

Ensuring quality during localization

How engineering at Calendly uses pseudo-localization techniques for testing.

Engineering at Calendly

This year, our team decided to take the full Calendly platform and localize it to multiple languages so that our customers can get the best experience possible no matter where they live or what languages they speak.

Localizing an application customizes the experience for users in different regions. A large portion of the work is in translating strings to different languages based on the locale of the user.

From a maintenance perspective, we didn’t want to have a completely new version of the website for each language. We decided the most effective way for us to translate the copy would be to replace each copy “string” of the site with a variable representing what the text is, and then replace the variable with the correct translation listed in a file – a process we call extraction.

We built translation files (YAML files) where each file has the list of keys and the translated text. The first file we extracted to was en.yml, since the application is already in English.

For example, on the left is en.yml (English) and on the right is es.yml (Spanish).

Just extract it. Simple enough, right?

We realized that extracting strings throughout the application code would be an undertaking. We calculated we had over 40,000 words across our application, help center articles and outbound emails. All of those strings of text would need to be extracted and we had to ensure that we didn’t miss anything.

So we asked ourselves, how can we verify that we have extracted everything correctly?

When engineers start extracting strings one by one from a page, everything that is ‘missed’ will still display in English. This means nothing will look different to the developer or QA—making it impossible to know whether you caught every string in your application. Even once you add translations, missed extractions can be subtle. Can you see the missed extraction on this page?

Image should be “Imagen” but missed extractions like this can be difficult to catch.

Finally, if you are starting with English, you will often find that other languages use more characters to convey the same message which might reveal UI issues from this longer than expected text. For example, a label on a form might look fine in English since it fits on one line, but in a different language, the label may break into two lines.

Here our CSS rules didn’t accurately accommodate the longer text and need to be updated.

Pseudo-localization to the rescue

Pseudo-localization is a testing practice where we create a new language file: pseudo.yml. We replace the text of the application with an altered version of the original language version.

This enables us to quickly get feedback and verify our changes (that text is actually extracted, the correct text shows, it’s resilient to length changes, etc.) instead of relying on the slower process of actual translation and language verification.

Our implementation was highly inspired by Shopify’s pseudo-localization gem but we needed some customization so we rolled our own version.

  1. We modified the translation process to correctly handle our interpolation. With Rails we use the format: “Your meeting with %{name} has been canceled.” We want to ensure that `%{name}` is not translated.
  2. We also wanted to exclude a few particular paths in our yml files. Our localization yml files contain values such as time format strings that we never want to be pseudo-translated. This enables us to have different date/time format strings for different locales that still work in pseudo mode.
  3. Finally, when we are in our development environments, we use a Rails initializer to trigger the creation of our ps.yml file with the pseudo-translated text.

Let’s translate!

Even with pseudo-localization in our tool belt, we still have to do additional work for each language to make sure that we are rolling out a high-quality experience. We do additional manual testing and proofreading to validate actual translations as well as any UI bugs that were not caught through the pseudo-localization process.

 

Have you localized an app? Comment below and tell us about the challenges you experienced. How did you address them?