Improved add-on localisation (translation)

It’s a lot of work for both developers and translators to ensure that an add-on is correctly translated into multiple languages but it’s a vital task for any add-on or software that will be used outside of the developer’s home country.

We want KeeFox to be available in many more languages but the existing Firefox localisation system does not do as much as it could to help us reach this goal.

This post will outline the changes I’ve made in KeeFox 1.2 in order to improve the localisation architecture for KeeFox and explain how other browser add-ons could use the same approach.

As I see it, these are the main problems with the standard Firefox localisation system:

Duplication	There are two distinct sources of text. In order to display a particular word or sentence in more than one part of an add-on it is sometimes necessary for that text to be translated twice. This can lead to mistakes and confusion, not to mention the extra workload it places on translators. This duplication is not a problem for simple add-ons that are created using recent improvements in the add-on infrastructure but many complex add-ons like KeeFox can’t be re-written in this new way (for both technical reasons and the shear amount of time it would take).
Fragility	Every language must have every piece of text translated or the entire add-on fails to function. This places a huge burden on translators and in add-ons such as KeeFox where there are many thousands of words to translate it can be a daunting prospect to start a new translation. From a developer perspective, this limitation leads to much manual work before each release in order to ensure that only the correct and complete translations are pushed out to the add-on users. Thorough testing of this manual process is also difficult and laborious.
Rigidity	Translation updates can only occur when the add-on code is updated. Even a one character typo correction has to pass through the full Mozilla add-on review process which often takes several weeks and distracts their review team from more useful tasks. (Note: the review team does not actually look for text accuracy - it’s mainly a check for basic add-on security and functionality). This also causes other issues such as unnecessary update installations for existing users and delays to the release date due to either waiting on ill or otherwise absent translators or the amount of work required to re-integrate the latest locale information every time.

To begin addressing these challenges I’ve developed a new locale system for KeeFox 1.2. It’s implemented as a JavaScript module so should be easily transferable to any other add-on.

Main features

• All locales are defined through .properties files.
• XML DTD files are not needed, even for localisation of XUL.
• If a translation for a particular string in the user’s current locale is not found, the “default” string will be displayed.
• In keeping with Mozilla standards, the “default” locale is “en-US” (although this would be easy to change if required).
• Locale definitions can be loaded after the application has started.

So this means

• No more XML DTD files.
• It works with existing Firefox property files.
• It works with dynamically loaded JSON locale files.
• It could work with existing Google Chrome JSON files:
  • Simple “messages” will work provided they are supplied through JavaScript rather than a separate locale file on disk.
  • Parameters won’t work yet (mainly because Chrome takes a different approach to parameter selection but also because it’s not been a priority for me to implement that feature yet).
  • See http://developer.chrome.com/extensions/i18n-messages.html for more details on the format of locale that could be supported in future.

What this means for KeeFox translators

• I’ll still be looking for 100% completion for each language included in the main KeeFox releases. The flexibility of the new system will provide protection against the mistakes that have previously caused entire countries to be unable to use KeeFox for several weeks.
• Temporary exceptions to the “100% complete” rule can be arranged as required.
• Translation can occur more independently of the release cycle of KeeFox.
• Languages do not have to be completed in order to be included in beta releases. This should help separate the task of translation from the testing phase of a KeeFox release and allow new translators to get their work seen before the entire translation process has been completed.
• The messages delivered by FAMS will continue to default to the existing English language and continue to be updated only when a KeeFox update occurs. Any translations added to the FAMS.keefox.properties file in Babelzilla will be included in the beta and final releases; 100% completion of this file is not required (although obviously I suspect that users will benefit if you can translate the whole lot).
• If I ever get around to implementing the “dynamic delivery over the internet” for FAMS messages there could be rare ad-hoc translation work (e.g. translating a security warning message) but I’ll provide further information on that when required.

Remaining problems

1. Having to manually construct a call to internationaliseElements() in each XUL file is messy. It should be possible to change this so that elements to be translated can instead be tagged with a class (or similar).
2. Dynamic update delivery: The localisation architecture allows for instant updates from the internet but there is currently no infrastructure in place to deliver this.
3. Dynamic update tools: The localisation file format is simple (JSON) but the existing Babelzilla translation does not currently support it so a temporary workaround will be required.

If you want to address any of those limitations please feel free to fork KeeFox and submit a pull request with your improvements.

Hopefully Firefox will release improvements to the core localisation system one day which will address the limitations that led me to develop an alternative solution for KeeFox. In the mean time, I thought other add-on developers might be interested in this work - documentation on its use is here: https://github.com/luckyrat/KeeFox/wiki/en-|-Technical#wiki-locales

KeeFox 1.2 experimental build ready for testing

Please test out the latest version of KeeFox. There are over ten thousand lines of changed code so I’d like to get some wider testing from willing users before pushing the new version onto the development (beta) channel.

The biggest changes are to form detection and form field matching which I’m sure you can guess is a fairly large chunk of the add-on. The main benefits are:

Improved form matching and filling

• Forms no longer need a password field (this feature is powered by a set of white/black lists which will need refinement during this testing phase and with each future KeeFox release)
• More multi-page logins work correctly
• New types of form field dealt with correctly (HTML5)
• The old “monitor each web page for new forms” feature has been upgraded to allow monitoring on only specific websites (if you used the old feature, you shouldn’t notice any difference but I strongly recommend disabling this on all sites except for the occasional specific ones on which it is needed)

Per-site configuration allows:

• You to quickly get many previously problematic websites working correctly
• KeeFox developers to make most popular websites work for all users in a future KeeFox version (to start with we’ve got Microsoft live.com working which is the most frequently mentioned “problem site” at the moment)

New localisation module reduces work for developers and translators (hopefully leading to new translators joining the team) and allows the “useful tips” and similar messages to be translated for the first time. In the next few days I’ll publish a separate article with more background and detail about this work.

A new configuration storage system within each KeePass entry is required to enable some of the features above so once you upgrade your KeePass database you won’t be able to go back unless you revert to an earlier backup. The upgrade process has been tested (and applied to my personal database with 500 entries) but there may be edge cases I’m not aware of so please pay attention to the warning about making a backup.

Please use the new manual on the github wiki and the github issues tracker instead of the Sourceforge Trac manual and ticket tracker that we’ve been using for several years. I’ll announce this change to everyone in due course but it’ll be simpler if I wait until version 1.2 is released because it contains some information that doesn’t apply to KeeFox 1.1.

If you have a backup of your existing database (if not, why not?) please download version 1.2.0a1 from the usual experimental XPI location.

Please post on the forum if you have any problems or comments (and make sure to state that you’re using v1.2.0a1).

KeeFox 1.1.2 released

The recently released KeeFox 1.1.2 contains dozens of new and improved features and a few important bug fixes, the highlights are below.

• Multiple database support. Seamlessly work with more than one KeePass database thanks to these improvements:
  • Logins from all open (unlocked) databases are matched (and automatically filled / submitted depending on your configuration)
  • Logins from all open databases are listed on the Logins toolbar button
  • A new login can be saved into any open database
  • Changing databases from within KeeFox is now reliable and predictable
• Improved form matching accuracy: when KeeFox knows how many form fields to expect for an entry it will now use that information to help select the correct form on a page
• Instant edit feature added: Review and edit every entry that KeeFox saves (disabled by default).
• Memory leak fixed: This will be a big improvement for heavy Firefox users who do not shutdown Firefox for days or weeks at a time
• Added option to require more accurate URL matching for specific entries (previously possible through a hidden KeePass “advanced string”)
• Thunderbird support (in beta testing)
• Improvements to setup instructions, especially for non-admin users and Mono (Mac & Linux) users
• First-run improvements (e.g. automatic save of your first KeePass database and a link to information about importing existing passwords)

You should get 1.1.2 pushed to you automatically unless you have disabled Firefox add-on automatic updates. You can manually install it from the main Add-on page.

Known limitation: Newsgroup authentication prompts in Thunderbird do not work with KeeFox.

Apple Mac support: The immaturity of the Mono platform that Mac users require to make KeePass work is keeping KeeFox Mac support in beta testing for the time being. You can install the development version if you’d like to give it a try (it seems to mostly work if you have some patience and know what you’re doing)

Known bug: Shortly after the release of 1.1.2 I learnt that users who enable the Firefox master password (for use by other add-ons, Firefox Sync, etc.) sometimes experience problems entering the master password when prompted before Firefox has completely started. I have put a fix into version 1.1.3 which you can download from here: https://github.com/luckyrat/KeeFox/blob/ … d79301bb/XPI-package/latest.xpi?raw=true.

1.1.3 is an experimental build - although it has very few changes since the well-tested 1.1.2 it has not been thoroughly tested itself so I advise only installing it if you need to. I’ll monitor its usage and consider releasing it as an official tested version if it looks like enough people are affected by this bug.

I’ll be starting work on version 1.2 in a few weeks time so do let me know through the support forum if you find any bugs that I should be working on for that release. In the mean time, enjoy version 1.1!

KeeFox 0.9.5 and Linux/Mac support

Version 0.9.5 has just been submitted to Mozilla for review so will hopefully be winging its way to you through the usual update channel within a week.

Changes this time:

• HTTP Auth login boxes now correctly state when no matching passwords were found
• New domain matching rules prevent IP addresses from matching incorrectly
• Improvements to message/tip display (easier to prevent the same message appearing again once you’ve seen it)
• Miscellaneous changes to meet recent Firefox add-on best practice guidelines
• Dutch translation added

Also note that (thanks to krbvroc1) you can now run KeeFox on Linux and Mac systems but we’ve not had enough beta testing feedback to enable support outside of the development channel. In fact to my knowledge, it has not ever been installed on a Mac, although in theory it “should work”.

Please install version 0.9.5b if you want to help us test KeeFox on Linux or Mac systems. There are no significant changes between version 0.9.5b and 0.9.5 so you’re not missing out by moving to the development channel.

More details are available in the release notes for 0.9.5b (look a little bit down the versions page)

0.9.6b will be pushed to the development channel in a couple of weeks so if there are any important changes that need to be made to improve Linux and Mac support it would be great to hear about them soon.

KeePass 2.18 compatibility

Thanks to everyone that notified me of the problems with the recently released version of KeePass.

I hope to release a new version in 4-6 weeks with a beta available within a few weeks. In the mean time, the existing version of KeeFox will continue to work with KeePass 2.17. As with the previous problematic KeePass upgrade, your upgrade route will be much simplified if you continue using KeePass 2.17 until after you are running a new version of KeeFox*. There are currently no known security problems with KeePass 2.17 that necessitate an immediate upgrade.

This time there were many changes to KeePass that prevented KeeFox from working. I was notified of at least some of these changes in advance of the release of KeePass 2.18 but did not have a development environment to work and test with until after the official release last week.

I have identified a way to get KeeFox to work with 2.18 and most users shouldn’t notice any differences but in light of the following major changes to the KeeFox code I have decided to push the fix to existing users quite cautiously - there’s no point in risking breaking KeeFox for everyone just so that some people can run the very latest version of KeePass.

For anyone interested, the two major changes to the KeeFox code are:
1) I’ve upgraded (and then customised) the Jayrock JSON-RPC server bundled within KeePassRPC
2) I’ve set .NET 4 as the default runtime for KeeFox users who don’t yet have a copy of .NET installed

This work required an upgrade to Visual Studio 2010 and I took that opportunity to make a clean break from the existing SVN revision control at Sourceforge; KeeFox code is now available on github, details will be announced in the coming weeks.

* Or at least downgrade to 2.17 before installing the new version of KeeFox.