Clustory: reinventing the tabs in the browser (Mozilla Concept Series) from Staś Małolepszy on Vimeo.

In the video above (slightly too lengthy, I admit), I try to explore a different approach to managing visited pages. I suggest using the loads of data stored in the browsing history (mining for patterns!) and present it in a way that exposes two new dimensions: time and tasks.

The clustory screen (a portmanteau word from “cluster” and “history”) would present a timeline of the visited pages, organized by themes, topics, or actions. Each horizontal line represents a cluster, and the pages belonging to the clusters are portrayed using screenshot thumbnails or favicons. At the top, there’s a timeline indicating the time of visits, and giving some other contextual information like the weather or even (not illustrated in the mock-up) wi-fi hotspot names (SSIDs), which you could identify as your own, your company’s or your school’s.

The sidebar on the right gives you some details about each cluster, such as:

• a user-given name,
• most important keyword as identified by the clustering algorithm,
• last activity time,
• total time spent on the pages belonging to the cluster,
• action buttons: forget, bookmark, share (all pages in the cluster).

The page that you were viewing most recently is always the biggest thumbnail on the screen, to make it easy to click on it and leave the clustory screen. Moreover, it never disappears from the screen, even if you pan to see older visits or clusters that are below the page fold. Its meta-information is always visible, too. To get more information about other clusters, just hover over them, to expand them and show bigger thumbnails, as well some additional metadata.

You can drag and drop pages across clusters, for example when the algorithm isn’t accurate enough and made a mistake. Dropping a page into a new cluster would make the algorithm recalculate the probabilities of each word indicating that a new page belongs to it. Pages currently in the cluster would stay in it.

Which brings me to data mining. I believe that it would be sufficient to represent web documents as unordered collections of words that occur in them (the bag-of-words model). You could even go as far as checking only if a word is present in the document, without counting its occurrences. This would lead to a very simple binary representation that could then be used by a Naive Bayes classifier. The advantage of such approach is that we would be storing minimal amount of data per visited page and we wouldn’t need to recalculate the weights for each word and each document after opening a new page (as using an idf-based weight would require).

Please watch the video, download the Keynote presentation and share your thoughts in the comments. Thanks!

Today, let’s discuss the compatibility criterion, which I believe is the most important guideline for localizers to consider.

The compatibility criterion: Services included in Fennec must render correctly on mobile devices, assuring good readability and ease of interaction.

The compatibility criterion is important, because it allows us to base on Firefox’s guidelines and adapt them to the mobile context. In other words, Fennec’s productization should follow Firefox’s guidelines, making exceptions when the compatibilty criterion is not met.

As you probably remember, there are 5 productization components in Firefox, mapping to components in Fennec (in fact, the future of the live bookmark and the getting started page in Fennec is uncertain, but let’s just assume for now that they’re in):

1. Content handlers (a.k.a feed readers)
2. Protocol handlers (currently: webcal, mailto, irc)
3. Search engines
4. News feed (a.k.a. Live Bookmark)
5. The ‘Getting Started’ page

Fennec should ship only with services that meet the compatibility criterion. We should not promote websites that render badly on mobile devices, are impossible to read (e.g. because of the broken layout not adapted to the small screen), or impossible to interact with on mobile devices.

The good news is that I don’t expect there are many websites that don’t conform to this guideline. After all, Fennec’s developers have gone to great lengths to make sure the browsing experience is as good as on a desktop browser. You can pan and zoom easily, increasing the text size and focusing on chosen part of the website.

But I can imagine a couple of scenarios effectively breaking the mobile user experience, for example:

• a Flash-based ad covering a significant area of the website which is difficult to close/discard (due to very small close button),
• a non-standard input element with no accessibility fallback, which makes typing text into it difficult or impossible,
• websites detecting a small-screen device (which as a practice is not bad) and offering a broken layout.

I’m sure you can think of more example like these. Feel free to add them in the comments.

Mozilla’s mission ensures that the Internet, desktop and mobile alike, evolves in the  correct direction. In this context, that means open, participatory, and innovative. Good for the users. With Fennec, we’re becoming reponsible for building mobile Web the right way. We have the momentum and power to make an impact and make sure web designers and developers don’t ignore mobile in their projects. Because with no doubt, mobile will become one of the most important contexts of using Web, independently of region or demographics. Even more so, mobile is gaining a lot of traction in developing countries, which skip directly to the age of personal and portable computing. I like to think that we’re responsible for the direction in which the mobile Web evolves: open, participatory, and innovative.

Please add your thoughts. Is this criterion too strict? Too vague? Just right?

* * *

In the next post I’ll try to summarize my thinking about the search engines in particular and suggest a couple of categories of search engines that we might want to ship Fennec with.

I immediately liked the idea (in fact, I’ve been thinking about something like this since I first experimented with Exhibit, but never got around to learn more about the Quick Search syntax and buglist.cgi options) and created a very simple search command for Ubiquity to be able to access Andrew’s tool quickly and seamlessly.

I then started to experiment with different queries as I continued to work, so that I can test both the Quick Search and the Exhibit interface on real-life examples. This has been such a great help to me already, and it hasn’t been even a day since I started using it.

Up until now, to quickly get a status of a locale or a bug, I’d simply start typing the bug’s summary in the awesome bar and then go directly to the bug (if there was only one that I was looking for) or to the dependency tree of the locale’s release tracker. However, the dependency tree doesn’t give much information (only summary, number and status), which is not enough when you’re using keywords (e.g. “fixed1.9.1″) in open bugs to indicate that the bug was fixed on branch but not yet on trunk. (I wrote about it before.) Bugzilla’s Quick Search is great help in this situation, because it lets me be more precise about which bugs I want to see. However the default buglist.cgi view isn’t very useful to me in this scenario: it’s just a flat list of bugs, whereas I’m interested in grouping bugs by locale, at the very least. Enter BugXhibit, which allows you to slice and dice the search results to your liking.

One other thing that made this so helpful for me and that we introduced a couple of weeks ago, is the “productization” keyword in bugzilla.mozilla.org. We use this keyword for all bugs related to productization of Mozilla software, i.e. third-party services that extend its functionality (search engines, protocol and content handler, live bookmarks etc.). This allows me to see all open productization bugs with a query as simple as “!productization“, and Andrew’s BugXhibit lets me further manipulate this data by selecting bugs for a specific locale only (in “Components”), bugs that havn’t been yet fixed on branch (select only “productization”, not “fixed1.9.1, productization” in “Keywords”) or bugs that only regard protocol handlers (”protocol” in “Text Search”).

Since I wanted to tweak the interface a bit, I ended up cloning BugXhibit’s repository and making some small changes.  Namely, I set the ordering criteria of the Tile view to “Component” first, “Bug status” second and “Keywords” third. I chose component as the first criterion because in bugzilla.mozilla.org, under the “Mozilla Localizations” product, components translate to locales. I’m pretty sure I could use SimileAjax.parseURLParameters() to specify the ordering in the URL and somehow tell Exhibit to use it in the Tile view. I just don’t know yet how to do that :)

I uploaded the code to the l10n server, go ahead and try it if you wish: bugXhibit-st.

Yet again Exhibit turned out to be a very interesting project and a great tool for sketching prototypes quickly. It has a great potential for creating dashboard-like interfaces (e.g. we currently use it on the l10n dashboard) and I’d like to experiment with it more in the near future, while working on the new l10n dashboard that’s being developed.

I’m sure I’ll continue using BugXhibit, especially through the Ubiquity command. Thanks a million, Andrew!

For development, the process takes place simultaneously on several branches. For example right now, we work on trunk (mozilla-central, Firefox.next), 1.9.1 (releases/mozilla-1.9.1, Firfox 3.5) and 1.9.0 (CVS trunk, Firefox 3.0.x). When a bug receives a patch, it first lands on trunk, and the bug is marked as fixed. Then it lands on branch(es), after getting needed approvals, and the bug is given the fixed-* keywords, for instance, fixed1.9.1.

I have a feeling that this workflow doesn’t fit well into the localization process. The localization efforts often focus on only one branch. In the current case, that’s 1.9.1. Some localizers like to keep their 1.9.1 and central repos in sync, and that’s great, but we don’t require them too. If they prefer working on 1.9.1 only, that’s perfectly fine. After all, strings on trunk have higher probability of changing, so it’s understandable that someone prefers to wait until they stabilize. I.e. until they land on a more stable branch.

So translations land first on 1.9.1, and for many localization, they will only be pushed to central after the 3.5 release, at which point central will no longer be central, but will be given its own branch. The process will start over: the localization efforts will focus on this branch, because this is where the next major release of Firefox will come from.

As you can see, for localization the workflow is reverse: we first translate on the more stable branch, and then port over to the dev branch (trunk). And this is what made me start thinking whether we should change the rules of marking the bus as fixed.

Why is this a problem? Let’s take Assamese as an example (just because it illustrates my point quite well and I’ve been working on it recently). Bug 443139 is the Firefox 3.5 release tracker for Assamese and it’s blocked by a couple of bugs that will need to be resolved before we can release Assamese version of Firefox. In fact, most of them have been already resolved, so we’re really close to releasing Assamese. Congrats to the Assamese team :)

If you look for example on the “Firefox 3 RSS reader setup for Assamese” bug (bug 443145), you’ll notice that it was marked as fixed and has a fixed1.9.1 keyword. That’s because it was fixed on both trunk and branch. So the bug’s status is now “resolved fixed” (thanks to the landing on trunk), and it doesn’t block the release tracker anymore.

But now take a look at the “Search engine setup for Firefox 3 for Assamese” bug (bug 443142). The fix landed on 1.9.1 only, so the bug has the fixed1.9.1 keyword, sure. But since it hasn’t been yet fixed on central (for Firefox.next, mind you), the bug’s status is “new”, and it is still blocking the 3.5 release tracker. Why the lack of landing on central should block a 3.5 release?

That’s a problem for me, because it makes using and following the tracker bugs harder. Of course the right question to ask here is “if we mark this bug as resolved fixed right now, how do we make sure that the fix lands for Firefox.next, too?”. I don’t have a good answer to this yet. But even in the current workflow, if the fix lands for Firefox.next after mozilla-central gets branched again, will we mark the bug as fixed, or maybe just add a fixed1.9.2 keyword? (or a different one, depending on what the name of the branch will be).

I don’t have any concrete proposals right now to address the problems described above. I feel that we should just resolve bugs as fixed when the fix lands on current next-major-release branch (1.9.1 these days). We should still use keywords to make it easier when looking back at fixed bugs to figure out on which branches they were fixed. Lastly, next time we branch from trunk, we should somehow make sure all the fixes are included. I’m not yet sure how, though.

What do you think? I wanted to share my thoughts on this out in the public. I’d be grateful for your comments!

• Добре дошли, хора!
• Planvenguts umans !
• मनुष्यांचे स्वागत आहे!
• (Welcome Humans!)

Many thanks to Ognyan (Bulgarian), Sandeep and Priti (Marathi) and Yannig (Occitan) for their continuous work on the quality of the localization and awareness of it among local users. The 3.0.9 release is a culmination of weeks and months of the efforts by these guys, both on the localization front (translation, productization, web parts, testing) and the promotional one (advertising the beta builds, getting feedback from beta users, increasing awareness of the browser etc.). Good job!

Thanks for all the bugs we fixed together. Looking forward to working with you again soon on next updates and versions of Firefox.

This race is only about to start :)

In the last post I announced a short series about productization of Fennec. This is the second post in the series.

Before we actually start talking about the default services and best choices, we should provide some background on why’s and what-for’s.

There are two main purposes of adding default web services to Mozilla products:

1. provide users with useful and relevant content, and
2. demonstrate certain features of the product.

The first one (provide users with useful and relevant content) is obvious: we want to improve our users’ experience on the Web, so we provide a couple of well-thought suggestions for different services. For example, we ship Firefox with 6 or 7 search engine plug-ins to make users’ lives easier when they’re looking for information, translation, products, multimedia, spelling and definitions etc. Another example: when the user clicks on a mailto: link, we suggest a couple of possible handlers chosen from the applications installed on their computer. But we also suggest a few web-based clients (in en-US these are Yahoo! Mail and GMail right now, Hotmail’s coming), so that if the user happens to use one of these in the browser, they don’t have to configure them.

The second purpose (demonstrate certain features of the product) is equally important: by providing these default services, we demonstrate particular features of the product, the ones which otherwise wouldn’t be as discoverable. For example, putting one sample news feed on the Bookmarks Toolbar in new profiles in Firefox helps in learning about the Live Bookmarks. Including a couple of search engines instead of one in the Search Toolbar can help users understand that they can add as many plug-in as they wish. Also the Getting Started page in Firefox was created with this goal in mind: to show users how to get the most out of Firefox and the Web in general.

Overall, in Firefox, the productization includes five areas:

1. A news feed (a.k.a. Live Bookmark),
2. News readers,
3. Protocol handlers (currently: webcal, mailto; irc coming soon),
4. Search engines,
5. The ‘Getting Started’ page.

We’re still no entirely sure what the list would be for Fennec, so for now, let’s focus on these 5 areas.

The guidelines for productization of Firefox can be found on the wikimo. Let me quote an important passage from that page:

We believe that localization teams are in the best position to provide recommendations on what local providers we can use for Web Services because you’re in the market, work in the language, and know your users.

The same thing is true for Fennec, or any other product for that matter. The guidelines that we are working on right now are supposed to offer helpful suggestions, evaluation criteria and rules for choosing default web services for your locale. Your feedback on these guideline will be greatly appreciated, because after all, you as localizers will be making suggestions for productization of Fennec is your locale.

The notion of the mobile context spans both the hardware and the situation in which the user operates. These are very different from what we observe and know from the desktop experience and Firefox. To name just a few key differences:

• screen: much smaller on mobile devices,
• user control and input: touchscreen, on-screen keyboard, finger instead of the mouse cursor,
• use cases: for example, quick reference look-up, price comparison, closest restaurant or ATM (in these examples the browser is used for an ad-hoc and quick information retrieval).

Just as Firefox ships with a set of default search engines, content and protocol handlers etc., which are adapted to the needs of a desktop user, so will Fennec. The ensemble of these settings is called the ‘productization’. The important thing regarding the web services that make up for the productization of Mozilla software is that they are often specific to a local market. For example, a search provider can deliver a very good quality of search results — but only in a specific country or language, and we have to take this into account when considering default search engines for a locale. This brings us to the need of customizing the set of default web services on a per-locale basis, in order to ensure a good user experience across all locales.

In a couple of next posts on this blog I will try to describe which factors should be, in my opinion, considered important when choosing default services for localized versions of Fennec. The series is intended as a discussion starter and I would like the guidelines laid out in it serve as a base for a conversation about the localization-related needs and issues with Fennec’s productization. As Fennec is still under development, all people involved in the localization process can make an important contribution to the project by bringing to our attention specific needs of their locale.

• Estonian (et)
• Kannada (kn)
• Telugu (te)

Congratulations to the localization teams that made it possible!

Seth already did two great posts about our process: So what does “getting new languages ready for release” actually mean? and Moving a locale out of beta, but I thought I’d give some more detail on how the process fits into our code freeze and release schedule.

When we first add a new localization to the tree, it’s in its pre-beta status (let’s call it alpha). This happens after an initial review of the localization by Gandalf or Pike (the review includes checking if the directory structure is correct, the encoding is OK, if all entities are used correctly etc.), at which point a flock of new bugs is filed by the l10n-drivers team, as described on the wiki. These are things like creating a new Bugzilla component for the locale, landing the localization in its brand new repository or modifying the all-locales file for each branch that’s ready. The locale’s nightly builds start to build.

Localizers then continue working on the translation, pushing changes to the repository. At this point the productization and the web parts processes start. The guidelines and the description of the productization process can be found on the wiki. Two bits of the productization process are given special priority at this point: search engine plug-ins setup and protocol handlers setup (e.g. web-based mailto: handlers). We try to find a good and final list of both of this productization bits before the first beta release of the locale, or, if deciding on the final list is not possible straight away, we try at least to only include search engines and protocol handlers that we are sure are final. This is due to the fact that changing them later in the process is hard: once a user installs a beta release, their profile is created with these settings and removing search engines and protocol handlers from the profile gets tricky. For these reason search engines and protocol handler setup are both called ‘beta blockers.’

When the localization reaches a level at which it’s ready to be reviewed by the community, it is given the beta status and is listed on Mozilla.com website under ‘beta locales.’ This is done by modifying the shipped-locales file, which should happen before a code freeze for the next release. (Technically, the changes to shipped-locales are NPOTB, which stands for Not part of the build, so it is possible to land them after the code freeze, but is not preferred.) We also need to modify the product-details files which manage which versions are offered to download on Mozilla.com (e.g. on the all.html page). This is done by the WebDev team and the bug needs to be filed a couple of days before the release.

At this stage we can start gathering feedback from the users testing the release. Feedback can be anything: spelling or grammar mistakes, better wording suggestions, font rendering issues, dialog sizes, better suggestions for productization (this is why deciding on the final list in the previous step is preferred –  it allows us to gather feedback on productization from users too) etc. This is a very important stage and the feedback that the localizers get from beta users helps improving the quality of the final release.

All fixes to the issues reported by the testers need to be landed in a timely manner before a code freeze. A couple of simple rules should be followed:

1. Changes to translation on the development branches can be landed freely.
2. Changes to productization on the developement branches additionally require a review (r+) before they can be landed.
3. On stable branches, all changes to localization (translation and productization) should be reviewed and approved by the l10n-drivers team.

For example for the 3.0.x branch, if a localizer wishes to modify the translation (e.g. fix a typo), we ask them to file a bug and request a review of the patch. The reason for this is that we want to make sure that 3.0.x builds are green at all times (in case of an emergency in which a ultra-quick build is required to address a security hole, for example). Hence the review&approval process for all l10n changes on the 3.0.x branch.

While the locale is in beta, we also continue to work on productization and web parts. If not chosen before the beta release, we can now choose a news feed that will be featured on the bookmarks toolbar as the live bookmark or select a couple of good and popular news readers that will be offered to the user when they wish to subscribe to a feed. We also focus on the Getting Started page and other web parts, managed by Pascal. Since the web parts are served from the Mozilla.com servers, no changes to the l10n code are required to publish them. Consequently, it is possible to work on them even after the code freeze.

During all this time, the l10n-drivers team meet at least once a week for a so called triage call (Tuesdays at 6 PM CET / 9 AM PST) when we monitor and discuss the progress of the de-beta-ing process. We go through open blocker bugs for each locale and update a wiki page on which we track the whole process (for 3.0.x that’s Triage:3.0.x). From time to time we e-mail the localizers to discuss next steps, ask about the current feedback and offer help. We’d like to do a good job in this field, so if you are a localizer of a beta locale, please give us your feedback on how we did in the comments below, by e-mail or via Rypple (it’s completely anonymous).

If statistics are available, we look at the usages figures (ADU numbers and downloads) to see if they are increasing. This is a good indicator that end-users are, at least generally, satisfied with what is available.  Finally, when the localizers feel good about their work, there are no more open blocker bugs, the feedback from the users is positive and the testing of the builds under all three platforms hasn’t showed any rendering/display/sizing/other issues, the decision to move the locale is taken. A couple of days before the release the product-details files are changed again to make sure that the locale will be listed as final, not beta (for 3.0.7 the bug for this was bug 480391). The release notes are changed accordingly. And when the release day comes, we party :)

UPDATE: Originally we also intended to de-beta Occitan (oc), which was on the list above, but it turned out that bug 454169 was still open. Stay tunned for 3.0.8! :)