Silly question about localizing C/C++ applications

[tranglos]

I've been looking at how the UI is translated in various apps, because I'm trying to figure out the optimal way of doing that in Delphi. I know of many Delphi ways, but none is too appealing, so I'm wondering how Real Programmers do it in C or C++. And now for my silly question:

Some C++ apps come with language files that look like this:

[Select]

HELLOWORLD "Hello, World!"

I'm curious: how is this loaded and interpreted? Is HELLOWORLD a string literal, and you have a long list of conditionals such as  "if id equals "HELLOWORLD" then sHelloWorld = id, else if..."? (That would be quite slow and it's one of the things I'm trying to avoid in Delphi.) Or do these strings map to numeric values somehow? Or something else yet?

Silly question!

[mouser]

There is no one standard solution, but the closest for c++ is probably gettext.

However, for C++ Builder (and of course it also works for Delphi), I have had reasonably good experience with the commercial product Korzh Localizer.

[tranglos]

There is no one standard solution, but the closest for c++ is probably gettext.

-mouser (January 03, 2012, 12:54 PM)

Ah, but this isn't what I was asking! I'm sorry to be such a nuisance The example in my OP was not from gettext, because gettext works directly with literal message strings, so it avoids using any kind of identifiers. Gettext .po files look like this:

[Select]

msgid "Done:"
msgstr "Concluído:"

And I am really interested in what happens with the lng files of the format

[Select]

MSG_ID "Hello, world!"

There is a gettext port for Delphi, but I don't like it for several reasons. One is that it uses pre-compiled language files, so testing is needlessly technical and time-consuming (and you need a lot of testing when translating an app). Second, it's based on the rule that one source string can have only one translation - whoever came up with this design may have been a good programmer but never was a translator, believe me. Going from English to Polish, "Open" needs one translation when it's a menu item and another one when it's a dialog box title. And it is not in any way special - in fact, it's the norm. So you will understand when I say gettext sucks big time

(Unless that last design principle has improved recently, that is. And yes, I know about the idea of "domains" in gettext. As I understand it, the person responsible for deciding what goes in which domain is the programmer. But the programmer doesn't know! The programmer cannot know that a particular term will need three different target forms in Farsi, depending on where they appear. Only the translator knows this. Gettext is silly that way.)

However, for C++ Builder (and of course it also works for Delphi), I have had reasonably good experience with the commercial product Korzh Localizer.

I know of Korzh, but it's deadly expensive, at least the source code edition. It's so expensive that I haven't even looked at it very closely yet. Could you tell me what format the language files are? Are they binary or human-readable text? Because I definitely want a mechanism where users can edit the language files themselves in a text editor, without any third-party software and without a compilation step in-between editing and testing the translation. (Plus, all these add-on translation tools are quite horrible and as a translator myself, I am quite sure their authors never actually used them to translate any app bigger than Hello world. Things like editing long strings in a ListView. Not gonna go that way.)

But I digress. What kind of (basic but efficient) loading mechanism would a C++ programmer use if he or she were not using any 3rd party libraries for the task?

[Stoic Joker]

Not sure if this'll help but there was a version of TClock prior to mine done by a guy named Two_toNe that used a string resource .dll for all the localized UI languages. It was really a pretty slick setup (that I should have used but didn't), and all necessary (pure C) source is included in the download.

Here's a link to a copy of the clock & source from my site:
http://www.stoicjoke...oads/tclock2_120.zip

[vlastimil]

There always are resource DLLs. I believe Microsoft still recommends them. They contain string tables (numeric_id=>"string value") and also dialog resources (allowing you to rearrange controls if they do not fit). You can have multiple different translations for the same phrase and you can have larger labels on dialogs if they are needed in a translation, you can have even different pictures or accelerator tables. Accessing the resources is fast. It all works perfectly - on the paper.

I have used resources DLLs in the past, but I have switched to a gettext-like method few years ago and it was the right move. With resource DLLs, you need specialized expensive software to do the translation, there are problems with updates and patches. Users are unable to contribute. I had people offering help with translation and had to turn them down, because the process was too complex.

With gettext-like method, there are limitations, but there are also results. My application was translated to multiple languages year after I made the switch. Granted, the translations were not perfect, but I bet the users are very happy they have a localized application with few weak points instead of just English app. If you are a one-man software shop, I would definitely stick with gettext or something very similar. If you have a lot of money to spend on a professional translation service and a long release cycle, you may want to try another, "more perfect" method.

Also, I would not worry about speed with a gettext method. If you put the strings in a map, it will be fast enough with thousands of strings.

[Eóin]

The the native MS approach is indeed resource DLLs as vlastimil suggests though the recommended way to use them is through Microsoft Multilingual User (MUI). I personally found this example, Hello MUI, great for getting up to speed with the API.

The FSF guys have long used gettext as mouser mentioned. Recently Boost added a localisation library, Boost.Locale, to their ever expanding collection. I haven't yet had a chance to use it, but looks neat. It seems to build on both gettext and ICU.

Some C++ apps come with language files that look like this:

[Select]

HELLOWORLD "Hello, World!"

I'm curious: how is this loaded and interpreted? Is HELLOWORLD a string literal, and you have a long list of conditionals such as  "if id equals "HELLOWORLD" then sHelloWorld = id, else if..."? (That would be quite slow and it's one of the things I'm trying to avoid in Delphi.) Or do these strings map to numeric values somehow? Or something else yet?

Silly question!

-tranglos (January 03, 2012, 12:38 PM)

Nope not silly, my guess is that the file is loaded into an associative array of some kind. In C++ that could be as simple as an STL std::map<std::string, std::string> container.

[tranglos]

Nope not silly, my guess is that the file is loaded into an associative array of some kind. In C++ that could be as simple as an STL std::map<std::string, std::string> container.

-Eóin (January 03, 2012, 03:26 PM)

Thanks, Eóin! A hash table totally makes sense, it's on the top of my choices right now as well.

I'm trying hard to figure out a way to do this in Delphi so that it is convenient both for the users of my apps and for myself. I've done that once before, an entirely home-made localization mechanism, still running in my password manager Oubliette. Now I need to update it for Unicode, but I'm trying to design something better. It's such a pain to write code when instead of saying

[Select]

showmessage( 'This is a test' );

I have to say

[Select]

showmessage( translate( msgThisIsaTest ));

...and always remember to add somewhere else

[Select]

resourcestring
  msgThisIsaTest = 'This is a test';

...and repeat it hundreds of times.

GetText wins here, because it does not require adding these identifiers all over the place; it works directly with string literals. But, apart from my earlier comments, I'd rather not use compiled language files, since they're a nuisance. Though I am considering it as well.

[tranglos]

There always are resource DLLs. I believe Microsoft still recommends them. They contain string tables (numeric_id=>"string value") and also dialog resources (allowing you to rearrange controls if they do not fit). You can have multiple different translations for the same phrase and you can have larger labels on dialogs if they are needed in a translation, you can have even different pictures or accelerator tables. Accessing the resources is fast. It all works perfectly - on the paper.

-vlastimil (January 03, 2012, 02:26 PM)

Resource DLLs are the built-in solution in Delphi as well, but they workflow they require is quite impossible for a one-person freeware shop. Send text to translator, get it back, compile, send back for testing, recompile... Whatever design I settle on, it will have plain text Unicode language files that anyone can edit freely.

Thanks for all the thoughts and pointers, everyone!