The Unicode Standard

The most important consideration when dealing with character sets across a system is ensuring that all characters that you plan to represent within the PeopleSoft system exist in the character set that is used by each component of the system.

For example, if you plan to maintain Japanese characters in employee names, you must ensure that:

• The character set that is used by the database system includes Japanese characters.

• Each external system feeding into or out of the PeopleSoft system expects data in a character set that includes Japanese characters.

• Workstations and printers are installed with fonts that include those characters.

For example, the Japanese Shift-JIS character set contains Japanese and many US-ASCII characters; it is sufficient for encoding both English data and the primary characters that are required in Japanese. However, it does not include the accented Latin characters that are needed for French, German, and other languages, so it is not a suitable character set for implementations that encompass Western European countries.

Given the sample list of common character sets in the previous table and the number of languages that are required by a typical global PeopleSoft implementation, selecting a character set can be daunting, especially when you are planning to support a large list of languages.

To simplify this situation, an industry consortium of vendors devised a universal character standard: the Unicode standard. This internationally recognized character standard represents every character that is required to write virtually every written language. The Unicode standard was developed and is maintained by the Unicode Consortium in conjunction with ISO. This standard shares the character repertoire with ISO/IEC standard 10646: the Universal Multiple-Octet Coded Character Set (UCS), also known as the Universal Character Set for short.

The PeopleSoft system uses Unicode throughout PeopleTools to simplify character handling. The PeopleSoft system allows the use of Unicode within PeopleSoft databases to enable you to maintain a single database with characters from virtually any language.

The Unicode standard and the ISO 10646 standard are available from their respective organizations.

See The Unicode Consortium.

See International Organization for Standardization (ISO).

Unicode Encodings

Unicode defines a code space of more than one million code points (or characters). Unicode code points are referred to by writing “U+” followed by the hexadecimal number—for example, U+0000, U+0061, U+FFFF, U+27741, and so on. To manage such a large repertoire of characters, Unicode defines multiple planes each comprising 65,533 code points, or character positions. Plane 0, covering the range of U+0000 to U+FFFF, is known as the Basic Multilingual Plane (BMP) and is generally sufficient for almost all modern languages. The other planes are intended to encode extended ideographic characters, archaic scripts, and other rarely used characters (such as advanced mathematical symbols). All characters from planes outside the BMP are known as supplementary characters.

PeopleTools fully supports the use of characters from the BMP only; supplementary character support is limited to display in the browser, storage in the database, and reporting output in BI Publisher. A tool that can be used to view Unicode character properties is http://www.unicode.org/charts/unihan.html. If a character is at codepoint U+20000 or higher, it is a supplementary character.

Several different Unicode encoding forms have been standardized based on two encoding methodologies. Unicode defines these two encoding methods: the Unicode Transformation Format (UTF), and UCS. An encoding maps the Unicode code points (or perhaps a subset of code points) to sequences of code values of some fixed size. In UTF encodings, the number in the encoding name indicates the number of bits per code value; in UCS encodings, the number indicates the number of bytes per code value.

Four common encodings of Unicode are widely used:

• UTF-32 — a 32-bit, fixed-width encoding; equivalent to UCS-4.

• UTF-16 — a 16-bit, variable-width encoding.

• UTF-8 — an 8-bit, variable-width encoding, which maximizes compatibility with ASCII.

• UCS-2 — a 2-byte, fixed-width encoding; a subset of UTF-16 supporting characters in the BMP only.

Other Unicode encodings—such as, CESU-8, Java’s Modified UTF-8, UTF-1, and others—have specific, and sometimes internal, applications and are not widely used for the interchange of information.

While the PeopleSoft system currently supports only the UTF-8 and UCS-2 encodings, the following table presents a brief comparison of all four common encodings:

Encoding	Description	Min. Bytes per Char.	Max. Bytes per Char.	PeopleSoft System Usage
UTF-32	The full, 32-bit (four-byte) encoding of Unicode. Each Unicode character is represented by a four-byte number. For example, the Latin small letter a character is represented in UTF-32 hexadecimal as 0x00000061. UTF-32 was formerly called UCS-4.	4	4	None
UTF-16	An extension of UCS-2 in which the application references characters on planes other than the BMP by combining two UCS-2 code units to designate a single, non-BMP character. UCS-2 is upward compatible with UTF-16 in that each UCS-2 character is also a valid character in UTF-16. However, UTF-16 allows characters outside the BMP to be referenced. These additional characters, known as supplementary characters, require two UTF-16 code units: a low surrogate and a high surrogate, together called a surrogate pair. When no supplementary characters are present, UTF-16 is identical to UCS-2.	2	4	None
UTF-8	A transformation of Unicode that encodes each character as one to four bytes, depending on which character is being encoded. All US-ASCII characters are encoded in UTF-8 as one byte, and this byte is identical to the encoding in US-ASCII. UTF-8 data is therefore backward compatible with US-ASCII data. All characters in the BMP are encoded as one, two, or three bytes in UTF-8. Characters in other planes are encoded as four bytes in UTF-8. UTF-8 has three main advantages: it is fully US-ASCII compatible, US-ASCII data can be considered as UTF-8 data, and it does not require that all characters use two or more bytes of storage.	1	4	The PeopleSoft system uses UTF-8 for serving HTML pages in the PeopleSoft Pure Internet Architecture and for inbound and outbound XML.
UCS-2	A 2-byte (16-bit) representation of each Unicode character. As such, it can reference only 65,535 code points and is limited to characters in the BMP.	2	2	The PeopleSoft system uses UCS-2 in memory for the Microsoft Windows development tools and for the application server.

Unicode Encoding Examples

This section includes Unicode encoding examples for the following characters:

Character	Unicode Code Point	Description
a	U+0061	Latin small letter a.
ñ	U+00F1	Latin small letter ñ.
€	U+20AC	Euro symbol

The following table shows the hexadecimal representation of these characters in each of the four Unicode encodings:

Unicode Encoding	Latin Small Letter a(U+0061)	Latin Small Letter ñ(U+00F1)	€ Symbol(U+20AC)
UTF-32	0x0000006	0x000000F1	0x000020AC
UTF-16	0x0061	0x00F1	0x20AC
UTF-8	0x61	0xC3B1	0xE282AC
UCS-2	0x0061	0x00F1	0x20AC