Staś Małolepszy

How to test beta locales using Litmus. A short guide.

This post was migrated from my old blog. Some links might not work.

Testing is an important step in the process of making a new localization ready for its debut as a final locale. It’s also a great way of improving the quality of localization when the locale is still in the beta status, as well as after it’s been published as final, for example prior to the release of a new version. In this post I’ll try to quickly explain how to test a beta localization of Firefox 3.5. (You can successfully use same instructions to test 3.6+ builds, if you wish. Just be sure to choose the right test run.)

Litmus is a great tool used by our QA community for testing purposes. With a wide range of test cases grouped in test runs, it’s probably safe to say that it covers all aspects and features of Mozilla products. You can read all about Litmus over at QMO, in the Litmus tutorial.

If you want to test your localization of Firefox 3.5 using Litmus, the best place to start is the 3.5 l10n test run. It’s a great way of giving your testing a structure, which will help make sure you cover all localization. It’s also very helpful when testing periodically—by following the test run, you’re sure not to miss anything that you tested last time.

To do a Firefox 3.5 l10n test run for your locale, go to the Firefox 3.5 l10n localizer test run on Litmus and sign in. You can also created a new account if you haven’t already. Or, go to and choose Firefox 3.5 l10n localizer test run from the list of active test runs. (You may have to click View all available test runs at the bottom of the page first.)

Next, fill in your Build ID, Platform and Operating System, and, most importantly, your locale. Click Submit configuration. On the following page you’ll see testcases subgroups available in this test run. Right now there are no testcases in the RTL (right-to-left) subgroup, so just click Submit in order to proceed.

As of this writing, there are 23 testcases to go through, and for each of them you can indicate a result: not run, pass, fail or test unclear/broken. Please leave a comment when a testcase fails or the instructions are unclear or outdated.

When you’re done (you don’t have to do all testcases at one time), just click Submit All Results. Congratulations, you’ve just contributed to the testing efforts to ensure best quality of your localization!

There are other test runs available on Litmus, many of them more detailed and more thorough than the l10n test run (e.g. the catch-all test runs). You’re encouraged to perform these tests to provide even more testing coverage for your locale, but generally they’re not required to move a locale out of beta.

You may also be interested in seeing how much testing has already been done for your locale. It’s simple. Just go to the Search Results page, choose your locale on the far right and hit Show Results. You may also want to specify the product, branch and platform, as well as the timespan (e.g. show only tests performed in the last 2 weeks).

If you’d like to have more information about how to use Litmus, be sure to check out the tutorial on QMO. If you have questions, don’t be shy and do ask us in #l10n or #qa.

Published on 03.09.2009

Staś Małolepszy

Thoughts about the Internet, the information society, Mozilla and human-computer interactions.

Latest notes