How to test your programs with Unicode characters in multiple languages on Windows 2000

Windows 2000 makes it really easy to enter Unicode characters from other languages using IMEs (input method editors) and the on-screen keyboard. You can use these features to test your programs and make sure they will work on other languages of Windows and with the full range of available Unicode characters. You can test with multiple languages by following a few steps:

Enable the system to read and write files in the languages you want to test with.
Enable keyboard layouts for the languages you want to test with.
Change your dialogs to use a new system font, MS Shell Dlg 2.
Enter characters into your app using the new keyboard layouts or the Charmap program.

Note that you must have a Unicode build of your app. If you try this with an ANSI build, you can only enter characters that are in the code page of the language of your OS. If you try entering other characters, they will either show up as question marks, or different letters that are missing diacritics.

Enabling foreign languages

The first thing to do is configure the system to read and write files in other languages. Log in as an administrator and open the Regional Options control panel. At the bottom of the first page, there is a checklist box with a list of languages, as shown here.

Check all the languages that you want to test with. For this example, check Japanese and Vietnamese, and click OK. Windows will install fonts, IMEs, and other necessary files and then reboot.

Adding foreign-language keyboard layouts

After the reboot, open the Keyboard control panel and click the Input Locales tab. Click the Add button and then pick a language in the Input Locale combo box. For this example, choose Japanese. The IME combo box should show "Japanese Input System (MS-IME 2000)" as shown here:

[Adding keyboard layouts - 30K]

Click OK. Click Add again and select Vietnamese, then click OK. Your Keyboard property sheet should look like this:

Click OK to save your changes. There should now be a language indicator icon in the tray, which will show the native language of the OS (for example, "EN" if you're using English Windows). Pressing Left Alt+Shift rotates your keyboard language among the languages you set in the Keyboard control panel. You can also left-click the language indicator to get a menu of all the installed languages.

Changing dialog fonts

Now that the system is set up to let you enter foreign-language characters, you need to tweak your dialogs so they can display the characters. Windows 2000 has a new system font name, MS Shell Dlg 2. This is not a real font, but rather a special name the system recognizes. When a dialog is created with this font, the system substitutes a font that is capable of displaying characters in all of the installed languages. (You can read about Windows NT font substitution in MSDN under Platform SDK\Windows Base Services\International Features\National Language Support\About National Language Support\Localization and the Shell Font.) By default this font is Tahoma. Once you set your dialogs to use this font, they will automatically be able to display any Unicode character from any installed language. Note that you'll need to edit your .RC files by hand, since you can't set the font to MS Shell Dlg 2 through the MSVC dialog editor.

Entering foreign-language characters

If you change the keyboard language to Japanese, you'll see the IME in the lower-right corner. Using the IME requires knowledge of Japanese to enter correct syllables (and teaching Japanese is a little beyond the scope of this article), so for the purposes of testing you can use Charmap to enter Japanese characters.

To run Charmap, click Start, point to Programs, point to Accessories, point to System Tools, and click Character Map. Select either the MS Mincho or MS Gothic font, and scroll the character list down until you see the Japanese characters. Double-clicking any character adds it to the edit box at the bottom of the window, and clicking Copy puts all the characters in the edit box onto the Clipboard. Here is what Charmap looks like after double-clicking three characters:

After copying the characters to the Clipboard, you can them paste them into any input field in your program.

Another way to enter characters is by using the on-screen keyboard, a new Accessibility feature in Windows 2000. The on-screen keyboard lets you click buttons with your mouse, instead of hunting for keys on the keyboard. To open the on-screen keyboard, click Start, point to Programs, point to Accessories, point to Accessibility, and click On-Screen Keyboard. The Vietnamese keyboard is shown here:

Other languages will differ, naturally, in the placement of some letters. For Vietnamese, the top row of keys enters diacritics and accented letters, as well as the two keys between P and backslash. To add a diacritic to a vowel, type the vowel and then the diacritic. For example, to enter the name Nguyen, type "Nguy" as normal letters. To enter the e (which has both a circumflex and a tilde), press the 3 key (for ê) and then press the 7 key to add the tilde.

When you want to use the on-screen keyboard, click in the window that you want to receive the keystrokes, and then click in the on-screen keyboard window. When you click buttons on the keyboard, the keystrokes will be sent to the window with the focus.

You can get the latest updates to this and my other articles at http://home.inreach.com/mdunn/code/

You must Sign In to use this message board.


Per page

How to save and display Greek Characters in text box
srimail09

20:57 23 Jun '09

Hi all, i am not able to display greek characters in text box Please help Thanks in Advance
Sign In·View Thread·PermaLink

Need to implement a language
shweta.shrivastava

22:53 2 Nov '06

I have a win32 based C++ application

It needs to read Khmer language text from xml file and display it in one of the columns in a list control.

The text would be Khmer alone or a string having both English and Khmer.

I can notice that Office applications display text in Khmer script correctly. We want to do the same in our application too.

Right now it shows Khmer script as boxes.

Is there a solution for this?

If anybody has any possible solution for this problem, please guide us in this regard.

Shweta Shrivastava

-- modified at 6:41 Friday 3rd November, 2006

Re: Need to implement a language
khmer

23:19 18 Mar '07

when you see boxes mean it's support unicode (khmer) already so you can change font to a font contain khmer characters
Sign In·View Thread·PermaLink

Display lang. characters????????????
Anonymous

3:19 27 Sep '05

Hi,
We are working on a file system to which we 've recently added the foreign language support(chinese korean russian and many more). My question is that we want to display the content of the file system and the file names are the language characters. DOS prompt do not support foreign language characters.
We 've installed the language support for individual character on the Window -XP. So in order to display the language characters we 've to write a windows application which can display the characters on a Window/Dialog or some editor capable of showing the characters .
Can u please suggest a suitable approach or any indications.

Thanks!!!
Reagrds,
Priyanka Ahuja

Re: Display lang. characters????????????
Michael Dunn

6:26 27 Sep '05

Not sure what you mean by "DOS prompt do not support foreign language characters". Filenames are stored in Unicode, so if you have the language support and its fonts installed (from the Regional ctrl panel applet) then the display will work (as long as you use a font like Tahoma that contains the characters - do not use MS Sans Serif).

--Mike--
Visual C++ MVP Cool

LINKS~! Ericahist | 1ClickPicGrabber | NEW~! CP SearchBar v3.0 | C++ Forum FAQ Magnae clunes mihi placent, nec possum de hac re mentiri.

Re: Display lang. characters????????????
acrao80

20:34 6 Nov '05

Hi mike,
we are making a vc++ application unicode compliant.I am changing the resource file to implement pseudo localisation purpose.Not all the characters are displayed (its not that a typical character set eg some Greek characters are displayed , others are not).But nowhere are ??? displayed.Also the font set for individual menus and dialogs was MS San Serif.I tried changing it to 'Arial Unicode MS'.Also changed the reg value of 'MS shell DLG 2' to 'Arial Unicode MS'. What should i do? note: i have put all L"" before all unicode strings.

Amit
"Opportunity is missed by most people because it is dressed in overalls and looks like work."

Re: Display lang. characters????????????
Michael Dunn

13:21 13 Nov '05

The process for dialogs is:
Change DIALOG to DIALOGEX in the .rc file
Change the font name to "MS Shell Dlg"
Add the DS_SHELLFONT style

Now keep in mind that you need to be using a font that contains all those glyphs. Tahoma should contain the common languages (at least the 30 or so that the OS is localized into).

--Mike--
Visual C++ MVP Cool

LINKS~! Ericahist | 1ClickPicGrabber | NEW~! CP SearchBar v3.0 | C++ Forum FAQ Actual sign at the laundromat I go to: "No tinting or dying."

Re: Display lang. characters????????????
vivian_qjh

21:21 12 Jan '06

i have this problem in SDI/MDI application, some controls can not display right string, but that no resourse about mainframe with your method. what can i do? Thanks! qjh
Sign In·View Thread·PermaLink

Display language characters??????
Anonymous

2:35 27 Sep '05

Foreign Language support
Alex Evans

19:13 7 Apr '05

Hello all

I am working on internationalization of an application and tried using some of the functions that I could see as relevant in this context:

1)IsValidLocale() returns always true, with my 2nd argument being LCID_INSTALLED
2)Tried also using the pair EnumUILanguages() and the associated callback required, BUT could not find anywhere an example of what this call back should have inside it….

Any help will be appreciated

Thanks
Alex

xilli I am facing a similar problem
Zephyr_BP

3:52 27 Dec '04

Hi All,

I found that caret movement in a rich edit box (1.0) when containing Japanese text is erratic. On investigation found that "normal" SendMessage messages like "EM_POSFROMCHAR" and "EM_LINEINDEX" are returning improper values.
The whole setup works OK when English text is used.

I found that the Japanese text being displayed in the rich edit control are not composed of a single character (i.e DBCS). But does it automatically follow that rich edit control (1.0) will not be able to follow the exact caret movement style?

Has anyone any idea about what exactly is going wrong?

Thanks in Advance

Zephyr
As the wind blows

japanese characters fall over
Anonymous

1:08 16 Oct '03

When testing japanese characters i see some that change there direction after i finish typing them - for example a "1" will roll over look like a "-" (it kind of falls over). I wonder if this is a bug in my software?
Sign In·View Thread·PermaLink

Re: japanese characters fall over
oniak01

20:40 15 Aug '04

If you activated Japanese IME on your Windows, "1" may be converted to like "-" (Kanji character) which means 'one' in Japanese.
Sign In·View Thread·PermaLink

western european characters input/output problems on win2k of east_asia version
xili

6:32 6 Jun '03

western european characters input/output problems on win2k of east_asia version

background:
the platform is win2k of chinese version.

DWORD lcid3 = GetSystemDefaultLCID(); // ==> 2052
DWORD lcid4 = GetUserDefaultLCID(); // ==> 2052
LANGID langid3 = GetSystemDefaultLangID(); // ==> 2052
LANGID langid4 = GetUserDefaultLangID(); // ==> 2052

problem 1: RichEdit

char buf[20];
SendMessage( Handle_of_RichEdit, EM_GETSELTEXT, 0, (LPARAM) (LPSTR) buf);

the two selected charaters in the RichEdit is "\xF6\xE4"(german characters);
but the content of buf is "\x20\x20\x0".

problem 2: TextOut

char buf[20];
buf[0] = '1';
buf[1] = 224;

LOGFONT lf2;

memset (&lf2,0,sizeof(lf2));
lf2.lfHeight = 36;
lf2.lfWeight = FW_NORMAL;
lf2.lfCharSet = ANSI_CHARSET;
lf2.lfOutPrecision = OUT_DEFAULT_PRECIS;
lf2.lfClipPrecision = CLIP_DEFAULT_PRECIS;
lf2.lfQuality = DEFAULT_QUALITY;
lf2.lfPitchAndFamily = DEFAULT_PITCH | FF_DONTCARE;
strcpy(lf2.lfFaceName,"Arial");

HFONT hf = CreateFontIndirect(&lf2);
HDC hdc = GetDC ( hwnd);

HFONT hold = SelectObject(hdc,hf);

TextOut(hdc, 20, 20, buf, 2 );

the character buf[1] on screen is the \.notdef of the font.

a unwilling resolution:
the problems disapear when I set the system default language
to "western and american" through control panel .

it seems like the win2k treat the character(>127) as
chinese character in stead of western european character,

there must be a way to correct them without modifying the platform system,
anybody help ?

How can I creat my own IMEs (input method editors)
Phan Van Long

6:53 21 Feb '03

Hi! I'm Phan Van Long, I from Viet Nam , i've learn "Han Nom" which is an very old language and "Nom" characters is look like chiness but they are not chiness now i going to program a software which let us type Nom characters into any editor software , I want to creat a IME like Microsoft's ones Please show me how can i do thank! Phan Van Long
Sign In·View Thread·PermaLink	1.67/5

Resource Files ?
Anonymous

7:34 14 Nov '02

Hi, Nice article, but unicode isn't supported on all OS versions ... Any idea how can I use an ANSI build and create resources in a language different from the list supplied by VC++ 6 ? (That is, the list of languages and dialects that is shown for example in the "Insert Resource Copy" dialog). Thanks.
Sign In·View Thread·PermaLink	2.00/5

Better use "MS Shell Dlg" with DS_SHELLFONT
uwph

0:30 22 Oct '02

"MS Shell Dlg 2" only works with Windows 2000/XP. For beeing compatible with Windows 9x it is better to use "MS Shell Dlg" with the DS_SHELLFONT (this is DS_SETFONT | DS_FIXEDSYS) This flag will use "MS Shell Dlg 2" in Windows 2000/XP

See http://msdn.microsoft.com/library/en-us/intl/nls_4qcn.asp[^]

Also you have to make sure, that you use DIALOGEX resources. Visual Studio saves dialogs as DIALOGEX only, if an extended style bit has been checked (Like "Context Help")

Nice to have.
Martin Marvinski

10:56 9 Jul '02

This is a good article, I'm working on a project that will be tested using info from your article.
Sign In·View Thread·PermaLink	2.00/5