en_US.UTF-8 Locale Support Overview

en_US.UTF-8 locale is the flagship Unicode locale in the Solaris 7 product that supports and provides multi-scripts processing capability by using UTF-8 as its codeset.

UTF-8 is a file system safe Universal Character Set Transformation Format of Unicode / ISO/IEC 10646-1 formulated by XoJIG of X/Open in 1992 and approved by ISO and IEC as Amendment 2 to ISO/IEC 10646-1:1993 in 1996.

The locale supports computation of all characters defined in Unicode 2.1 / ISO/IEC 10646-1:1993 with Amendment 1 to 5 in the locale. However, due to a limited set of font resources and the fact that few users intend to use all of the code point values, users of the en_US.UTF-8 locale will see only character glyphs from the following character sets:

• ISO 8859-1 (Latin-1)

• ISO 8859-2 (Latin-2)

• ISO 8859-4 (Latin-4)

• ISO 8859-5 (Latin/Cyrillic)

• ISO 8859-6 (Arabic)

• ISO 8859-7 (Latin/Greek)

• ISO 8859-8 (Hebrew)

• ISO 8859-9 (Latin-5)

• BIG5 (Traditional Chinese)

• GB 2312-1980 (Simplified Chinese)

• JIS X0201-1976, JIS X0208-1983 (Japanese)

• KS C 5601-1992 Annex 3 (Korean)

• ISO 8859-8 (Hebrew)

• ISO 8859-11 TIS 620.7573 (Thai)

The above coverage of the scripts has significantly increased in the Solaris 7 product, which now also supports the following range of scripts:

• Western/Eastern/Northern European

• Greek

• Turkish

• Cyrillic

• Simplified Chinese

• Traditional Chinese

• Japanese

• Korean

• Arabic

• Hebrew

• Thai

Since this locale is primarily for developers, it belongs to the developer's cluster of the Solaris 7 product. Thus, when you install the Solaris 7 product, you should choose the developer's cluster to install the locale on your system.

Exactly the same level of en_US.UTF-8 locale support is provided for both 32-bit and 64-bit Solaris systems.

Motif and CDE desktop applications and libraries support the en_US.UTF-8 locale. However, OpenWindows, XView, and, OPENLOOK DeskSet applications and libraries do not support the en_US.UTF-8 locale.

System Environment

Locale Environment Variable

To use the en_US.UTF-8 locale environment, make sure to choose the locale first. Be sure you have the en_US.UTF-8 locale installed on your system.

To use the en_US.UTF-8 locale environment

1. In a TTY environment, choose the locale first, by setting the LANG environment variable to en_US.UTF-8, as in the following C-shell example:

   system% setenv LANG en_US.UTF-8

2. Make sure that other categories are not set (or are set to en_US.UTF-8) since the LANG environment variable has a lower priority than other environment variables such as LC_ALL, LC_COLLATE, LC_CTYPE, LC_MESSAGES, LC_NUMERIC, LC_MONETARY and LC_TIME at setting the locale. See the setlocale(3C) man page for more details about the hierarchy of environment variables.

3. To check current locale settings in various categories, use the locale(1) utility. locale(1)

system% locale 
			LANG=en_US.UTF-8
			LC_CTYPE="en_US.UTF-8"
			LC_NUMERIC="en_US.UTF-8"
			LC_TIME="en_US.UTF-8"
			LC_COLLATE="en_US.UTF-8"
			LC_MONETARY="en_US.UTF-8"
			LC_MESSAGES="en_US.UTF-8"
			LC_ALL=

You can also start the en_US.UTF-8 environment from the CDE desktop. At the CDE login screen's Options -> Language menu, choose en_US.UTF-8.

TTY Environment Setup

To ensure correct text edit operation by a terminal or by a terminal emulator such as dtterm(1), users should push certain locale-specific STREAMS modules onto their Streams.

For more information on STREAMS modules and streams in general, see the STREAMS Programming Guide.

Table 4-1 shows STREAMS modules supported by the en_US.UTF-8 locale in the terminal environment:

Table 4-2 lists the 64-bit STREAMS Modules Supported by en_US.UTF-8.

Loading a STREAMS Module at Kernel

To load a STREAMS module at kernel, first become root:

system% su
Password:
system#

To determine whether you are running a 32-bit Solaris or 64-bit Solaris system, use the isainfo(1) utility as follows:

system# isainfo -v
	64-bit sparcv9 applications
	32-bit sparc applications
	system#

If the command returns this information, you are running the 64-bit Solaris system. If you are running the 32-bit Solaris system, the utility shows the following:

system# isainfo -v
	32-bit sparc applications
	system#

Use modinfo(1M) to be certain that your system has not already loaded the STREAMS module:

system# modinfo | grep eucu8modulename

If the STREAMS module, such as eucu8, is already installed, the output will look as follows:

system# modinfo | grep eucu8
89 ff798000  4b13  18   1  eucu8 (eucu8 module)
system#

If the module is already installed, you don't need to load it. However, if the module has not yet been loaded, use modload(1M)as follows:

system# modload /usr/kernel/strmod/eucu8modulename

This loads the 32-bit eucu8 STREAMS module at the kernel, so you can push it onto a Stream. If you are running the 64-bit Solaris product, use modload(1M) as follows:

system# modload /usr/kernel/strmod/sparcv9/eucu8

The STREAMS module is installed at the kernel, and you can now push it onto a Stream.

To unload a module from the kernel, use modunload(1M), as shown below. In this example, the eucu8 module is being unloaded.

system# modinfo | grep eucu8
89 ff798000  4b13  18   1  eucu8 (eucu8 module)
system# modunload -i 89

dtterm and Terminals Capable of Input and Output of UTF-8 Characters

The dtterm(1) and any terminal that supports input and output of the UTF-8 codeset should have the following STREAMS configuration:

head <-> ttcompat <-> u8euc <->  ldterm <-> eucu8 <-> pseudo-TTY

In this example, u8euc and eucu8 are the modules supported by the en_US.UTF-8 locale. Make sure you already loaded the STREAMS modules into the kernel as specified in the previous section.

To set up the above STREAMS configuration, use,strchg(1) as shown below:

system% cat > /tmp/mystreams 
ttcompat
u8euc
ldterm
eucu8
ptem
^D
system% strchg -f /tmp/mystreams

When using strchg(1) , be sure you are either root or the owner of the device. To see the current configuration of STREAMS, use strconf(1) as shown below:

system% strconf
ttcompat
u8euc
ldterm
eucu8
ptem
pts
system%

To revert to the original configuration, set the STREAMS configuration again as shown below:

system% cat > /tmp/orgstreams
ttcompat
ldterm
ptem
^D
system% strchg -f/tmp/orgstreams

Terminal Support for Latin-1, Latin-2, or KOI8-R

For terminals that support only Latin-1 (ISO 8859-1), Latin-2 (ISO 8859-2), or KOI8-R, you should have the following STREAMS configuration:

head <-> ttcompat <-> u8euc  <-> ldterm <-> eucu8 <-> u8lat1 <-> TTY

This configuration is only for terminals that support Latin-1. For Latin-2 terminals, replace the STREAMS module u8lat1 with u8lat2. For KOI8-R terminals, replace the module with u8koi8.

To set up the STREAMS configuration shown above, use strchg(1), as follows:

system% cat > tmp/mystreams
ttcompat
u8euc
ldterm
eucu8
u8lat1
ptem
^D
system% strchg -f /tmp/mystreams

Be sure that you are either root or the owner of the device when you use strchg(1). To see the current configuration, usestrchg(1) , as follows:

system% strconf
ttcompat
u8euc
ldterm
eucu8
u8lat1
ptem
pts
system%

To revert to the original configuration, set the STREAMS configuration as follows:

system% cat > /tmp/orgstreams
ttcompat
ldterm
ptem
^D
system% strchg -f  /tmp/orgstreams

Setting Terminal Options

To set up the UTF-8 text edit behavior on TTY, you must first set some terminal options using stty(1) as follows:

system% /bin/stty cs8 -istrip defeucw

Since /usr/ucb/stty is not yet internationalized, you should use /bin/stty instead.

You can also query the current settings using stty(1) with the -a option, as shown below:

system% /bin/stty -a

Saving the Settings in ~/.cshrc

Assuming the necessary STREAMS modules are already loaded with the kernel, you can save the following lines in your .cshrc file (C shell example) for convenience:

setenv LANG en_US.UTF-8
if ($?USER != 0 && $?prompt != 0) then
				 cat >! /tmp/mystreams$$ << _EOF
				 ttcompat
				 u8euc
				 ldtterm
				 eucu8
				 ptem
_EOF
				 /bin/strchg -f /tmp/mystream$$
				 /bin/rm -f /tmp/mystream$$
				 /bin/stty cs8 -istrip defeucw
endif

With these lines in your.cshrc file, you do not have to type all of the commands each time. Note that the second _EOF should be in the first column of the file. You can also create a file called mystreams and save it so the .cshrc references to mystreams instead of creating it whenever you start a C shell.

Code Conversions

The en_US.UTF-8 locale supports various code conversions among major codesets of several countries through iconv(1) and iconv(1).

The available fromcode and tocode names that can be applied to iconv(1) and iconv_open(3)are shown in Table 4-3.

For more details on iconv code conversion, see the , iconv(1) and iconv_open(3), iconv(3), and iconv_close(3) man pages. For more information on available code conversions, see iconv_en_US.UTF-8(5).

Script Selection and Input Modes

The en_US.UTF-8 locale supports multiple scripts. There are a total of eight input modes in the en_US.UTF-8 locale:

• English/European

• Cyrillic

• Greek

• Arabic

• Hebrew

• Thai

• Unicode Hexadecimal code input method

• Table lookup input method

English/European Input Mode

The English/European input mode includes not only the English alphabet but also characters with diacritical marks (for example, á, è, î, õ, and ü) and special characters (such as ¡, §, ¿) from European scripts.

This input mode is the default mode for any application. The input mode is displayed at the bottom left corner of the GUI application.

To insert characters with diacritical marks or special characters from Latin-1, Latin-2, Latin-4, Latin-5 and Latin-9, you must type a Compose sequence, as shown in the following examples:

• For Ä, press and release Compose, then A, and then "

• For ¿, press and release Compose, then +, and then -

When there is no <Compose> key available on your keyboard, you can substitute for the <Compose> key by simultaneously pressing the <Control> key and <shift-T> together.

For the input of the Euro currency symbol (Unicode value U+20AC) from the locale, you can use any one of following input sequences:

• <AltGraph> and <e> together

• <AltGraph> and <4> together, or

• <AltGraph> and <5> together

These input sequences mean that you press both keys simultaneously. If there is no <AltGraph> key available on your keyboard, you can substitute the <Alt> key for the <AltGraph> key.

The following tables are the most commonly used Compose sequences in Latin-1, Latin-2, Latin-4,Latin--5 and Latin-9 script input for Sparc.

To start these sequences, type <Compose> key and release it.

Table 4-4 lists the Common Latin-1 Compose Sequences.

Compose sequences defined in Table 4-3 are not included in Table 4-4..

Table 4-5 lists the Common Latin-2 Compose Sequences.

Table 4-5 contains the Latin-2 compose sequences.

Compose sequences defined in Table 4-3 or Table 4-4 are not included in Table 4-5.

Table 4-6 lists the Common Latin-4 Compose Sequences.

Compose sequences defined in Table 4-3 or Table 4-4 or Table 4-5 are not included in Table 4-6.

Table 4-7 lists the Common Latin-5 Compose Sequences.

Any compose sequences already described do not re-appear in this table.

Table 4-8 lists the Common Latin-9 Compose Sequences.

Cyrillic Input Mode

To switch to Cyrillic input mode from English input mode, press Compose c c. If you are currently in Greek input mode, first return to English input mode, then switch to Cyrillic mode.

The input mode is displayed at the bottom left corner of your GUI application.

After you switch to Cyrillic input mode, you cannot enter English text. To switch back to English input mode, type Control-Space. The Russian keyboard layout appears in Figure 4-1.

Figure 4-1 Cyrillic Keyboard

Greek Input Mode

To switch to Greek input mode from English input mode, press Compose g g. If you are currently in Cyrillic input mode, first return to English input mode and then switch to Greek mode.

The input mode is displayed at the left bottom corner of your GUI application .

After you switch to Greek input mode, you cannot enter English text. To switch back to English input mode, type Control-Space. The Greek keyboard layouts appear in Figures 4-2 and 4-3.

Figure 4-2 Greek Euro Keyboard

Figure 4-3 Greek UNIX Keyboard

Arabic Input Mode

To switch to Arabic input mode, type <Compose> + <g> + <g> from your current input mode. The input mode is displayed at the left bottom corner of your GUI application Once you switch to the Arabic input mode, you have to switch back to English/European input mode to enter English/European characters by typing <Control> and <Space> together.

Figure 4-4 Arabic Keyboard

Hebrew Input Mode

To switch into Hebrew input mode, type <Compose> <h> <h> from your current input mode. The input mode is displayed at the left bottom corner of your GUI application.

Once you switched into the Hebrew input mode, you have to switch back to the English/European input mode to enter English/European characters by typing <Control> and <Space> and together. You can also switch into other input modes by typing the corresponding input mode switch key sequence. The Hebrew keyboard layout is shown at following figure:

Figure 4-5 Hebrew Keyboard

Thai Input Mode

To switch into Thai input mode, type <Compose> <t> <t> from your current input mode. The input mode will be displayed at the left bottom corner of your GUI application.

Once you have switched into the Thai input mode, you have to switch back to English/European input mode to enter English/European characters by typing <Control> and <Space> together. You can also switch into other input modes by typing the corresponding input mode switch key sequence. The Thai keyboard layout is shown at following figure:

Figure 4-6 Thai Keyboard

Unicode Hexadecimal Code Input Method Input Mode

To switch into the Unicode hexadecimal code input method input mode, type <Compose> <l> <l> from your current input mode. The input mode is displayed at the left bottom corner of your GUI application:

To use this input mode, you need to know about the hexadecimal code point values of the characters. Refer to The Unicode Standard, Version 2.0 for the mapping between code point values and characters. To input a character, type four hexadecimal digits, for instance, 00a1 for Inverted Exclamation Mark, 03b2 for Greek Small Letter Beta, ac00 for a Korean Hangul Syllable KA, 30a2 for Japanese Katakana Letter A, 4e58 for a Unified Han character and so on. Users can use both uppercase and lowercase letters of A, B, C, D, E, and, F for hexadecimal digits. If you mistype a digit or two, you can delete the digits by using the <Delete> key or the <Backspace> key.

Table Lookup Input Method Input Mode

To switch into table lookup input method input mode, type <Compose> <l> <l> from your current input mode. The input mode is displayed at the left bottom corner of your GUI application.

Once you turn on the input mode, there is a lookup window showing multiple candidates of Unicode characters. You can choose any one of the candidates by moving your pointer and clicking the left button on your mouse. You can also select any one of the candidates by choosing a left-hand- side letter associated with each of the candidates.

Once you are finished using the current input mode, you can switch into other input mode by typing corresponding input mode switch key sequence.

Input Mode Switch Key Sequence Summary

Starting in the Solaris 7 environment, , users can switch from one input mode to another without any restrictions. The following table shows the input mode switch key sequences for each input mode.