57 Handling Non-ASCII Code on the BRM Server
Learn how to use character-encoding conversion-layer macros with languages other than English (that is, any non-ASCII character encoding) in your Oracle Communications Billing and Revenue Management (BRM) system.
Topics in this document:
About Character-Encoding Conversion
To work with different languages, BRM applications must use a character encoding that supports them. To support any Western European language or East Asian language, the macros described in this document are required. You must use the conversion macros with any BRM client, server, or Web application localization that is not written in Java. Without these macros, only the 7-bit ASCII encoding works.
Note:
These macros are not required for English language applications, but using the macros facilitates later translations.
BRM supports localizations using Latin 1 and some of the East Asian encodings for Japanese, Korean, Traditional Chinese, and Simplified Chinese only.
About Converting Multibyte or Unicode to and from UTF8
Figure 57-1 shows the relationships between the BRM applications and the character-encoding conversion layer:
Figure 57-1 BRM Applications and the Character-Encoding Conversion Layer
![Description of Figure 57-1 follows Description of Figure 57-1 follows](img/prg_int_api_conv_layer.gif)
Description of "Figure 57-1 BRM Applications and the Character-Encoding Conversion Layer"
If you are developing a command-line application or a third-party integration, use the macros that convert either Unicode or multibyte input strings to UTF8 and from UTF8:
The macros check the defined preprocessor directive (_MBCS and _UNICODE) to call the direct conversion macros and call the supporting function and macro.
If you are developing a BRM client application or working with multibyte or Unicode only, use the direct conversion macros to change the character encoding. See "About Converting Multibyte or Unicode to and from UTF8":
The macros are located in the Portal Communication Module (PCM) library. The header file is in pcm.h.
Direct Conversion Macros
Table 57-1 lists the functions available in the direct conversion macros.
Table 57-1 Direct Conversion Macros
Function | Description |
---|---|
Converts a multibyte character string to UTF8. |
|
Converts Unicode characters to UTF8. |
|
Converts UTF8 characters to multibyte. |
|
Converts a UTF8 character string to a Unicode string. |
Supporting Functions and Macros
Table 57-2 lists the supporting functions and macros.
Table 57-2 Supporting Functions and Macros
Function | Description |
---|---|
Determines whether a specified string is using UTF8 encoding. |
|
Determines the length of the multibyte string. |
|
Sets, changes, or queries some or all of the current program locale, specified by locale and category. |
Universal Macros
Table 57-3 lists the universal macros.
Table 57-3 Universal Macros
Function | Description |
---|---|
Converts translatable database data to UTF8. |
|
Calls PIN_CONVERT_UTF8_TO_MBCS when _MBCS is defined, and calls PIN_CONVERT_UTF8_TO_UNICODE when _UNICODE is defined. This macro is called whenever translatable data is retrieved from the database. |
PIN_CONVERT_MBCS_TO_UTF8
This macro converts a multibyte character string to UTF8.
Note:
You need to use setlocale before calling this macro.
Syntax
int32 PIN_CONVERT_MBCS_TO_UTF8( char *pLocaleStr, char *pMultiByteStr, int32 nMultiByteLen, unsigned char *pUTF8Str, int32 nUTF8size, pin_errbuf_t *ebufp);
Parameters
- pLocaleStr
-
Indicates the locale of the multibyte string input. The locale string argument can have following values:
-
en_US - US-English locale
-
"" - System default locale
-
NULL - Where LC_CTYPE is set to the appropriate locale before calling the macro
-
- pMultiByteStr
-
Points to the character string to be converted.
- nMultiByteLen
-
Specifies the number of characters to be converted in the string pointed to by pMultiByteStr. If this value is 1, the string is assumed to be NULL-terminated and the length is calculated automatically.
- pUTF8Str
-
Points to the buffer that receives the converted UTF8 string.
- nUTF8size
-
Specifies the size of the buffer pointed to by pUTF8Str.
- ebufp
-
A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.
Return Values
Table 57-4 lists the values returned by PIN_CONVERT_MBCS_TO_UTF8.
Table 57-4 PIN_CONVERT_MBCS_TO_UTF8 Return Values
SourceString pMultiByteStr | Number of Characters to Convert in Input String nMultiByteLen | Buffer pUTF8Str | Buffer Size in Bytes nUTF8size | Return Value |
---|---|---|---|---|
Null terminated |
Any number |
pMultiByteStr = pUTF8Str |
Any size |
0 (ERR) |
NULL |
Any number |
Any |
Any size |
0 (ERR) |
Any |
< -1 |
Any |
Any size |
0 (ERR) |
Any |
0 |
Any |
Any size |
0 (ERR) |
Null terminated |
-1 or > 0 |
NULL |
!=0 |
0 (ERR) |
Not null terminated |
>0 |
NULL |
!=0 |
0 (ERR) |
Any |
Any number |
!=NULL |
<=0 |
0 (ERR) |
Not null terminated |
Any number |
pMultiByteStr = pUTF8Str |
Any size |
0 (ERR) |
Null terminated |
-1 or > 0 |
!=NULL |
>0 |
Converted characters |
Not null terminated |
> 0 |
!=NULL |
>0 |
Converted characters |
Null terminated |
-1 or > 0 |
!=NULL |
>0 |
Converted characters |
Null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Not null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Error Handling
If pMultiByteStr and pUTF8Str are the same, the macro fails and the error buffer returns PIN_ERR_BAD_ARG. If the macro encounters an invalid character in the source string, the macro fails; it sets the return value to 0 and the error buffer to the respective error code as shown in Table 57-5:
Table 57-5 Error Codes
Returned Error Code | Value | Reserved Bit in ebuf | Error Description |
---|---|---|---|
PIN_ERR_BAD_ARG |
4 |
0 |
Bad argument |
PIN_ERR_BAD_LOCALE |
71 |
0 |
Invalid locale string |
PIN_ERR_CONV_MULTIBYTE |
72 |
0 |
Error in multibyte to UTF8 conversion |
PIN_ERR_NULL_PTR |
39 |
0 |
Empty string passed |
PIN_CONVERT_STR_TO_UTF8
This macro converts translatable database data to UTF8.
Syntax
int32 PIN_CONVERT_STR_TO_UTF8( char *pLocaleStr, char *pSourceStr, int32 nSourceLen, unsigned char *pUTF8Str, int32 nUTF8size, pin_errbuf_t *ebufp);
PIN_CONVERT_UNICODE_TO_UTF8
This macro converts Unicode characters to UTF8.
Note:
You need to use setlocale before calling this macro.
Syntax
int32 PIN_CONVERT_UNICODE_TO_UTF8( wchar_t *pUnicodeStr, int nUnicodeLen, unsigned char *pUTF8Str, int nUTF8, pin_errbuf_t *ebufp);
Parameters
Return Values
Returns NULL in the error buffer if the macro is successful. Returns the cause of the error if the macro fails.
Error Handling
If pUnicodeStr and pUTF8Str are the same, the macro fails, and the error buffer returns PIN_ERR_BAD_ARG. If the macro encounters an invalid character in the source string, the macro fails; it sets nUTF8 to 0 and sets the error buffer to the respective error code as shown in Table 57-6:
Table 57-6 Error Handling Codes
Returned Error Code | Value | Reserved Bit in ebuf | Error Description |
---|---|---|---|
PIN_ERR_BAD_ARG |
4 |
0 |
Bad argument |
PIN_ERR_CONV_UNICODE |
73 |
1 |
Error in UTF8 to Unicode conversion |
PIN_ERR_BAD_UTF8 |
75 |
0 |
Invalid UTF8 characters |
PIN_ERR_NULL_PTR |
39 |
0 |
Empty string passed |
PIN_CONVERT_UTF8_TO_MBCS
This macro converts UTF8 characters to multibyte.
Note:
You need to use setlocale before calling this macro.
Syntax
int32 PIN_CONVERT_UTF8_TO_MBCS( char *pLocaleStr, unsigned char *pUTF8Str, int nUTF8Len, char *pMultiByteStr, int nMultiByte, pin_errbuf_t *ebufp);
Parameters
- pLocaleStr
-
Indicates the locale of the multibyte string input. The locale string argument can have following values:
-
en_US - US-English locale
-
"" - System default locale
-
NULL - Where LC_CTYPE is set to the appropriate locale before calling the macro
-
- pUTF8Str
-
Points to the character string to be converted.
- nUTF8Len
-
Specifies the number of bytes to be converted in the string pointed to by pUTF8Str.
- pMultiByteStr
-
Points to the buffer that receives the converted multibyte string.
- nMultiByte
-
Specifies the size of the buffer pointed to by pMultiByteStr.
- ebufp
-
A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.
Return Values
Table 57-7 lists the values returned by PIN_CONVERT_UTF8_TO_MBCS.
Table 57-7 Values Returned by PIN_CONVERT_UTF8_TO_MBCS
Source String pUTF8Str | Number of Characters to be Converted in Input String nUTF8Len | Buffer pMultibyteStr | Buffer Size in Bytes nMultibyte | Returned Value |
---|---|---|---|---|
Null terminated |
Any number |
pMultibyteStr = pUTF8Str |
Any size |
0 (ERR) |
NULL |
Any number |
Any |
Any size |
0 (ERR) |
Any |
< -1 |
Any |
Any size |
0 (ERR) |
Any |
0 |
Any |
Any size |
0 (ERR) |
Null terminated |
-1 or > 0 |
NULL |
!=0 |
0 (ERR) |
Not null terminated |
>0 |
NULL |
!=0 |
0 (ERR) |
Any |
Any number |
!=NULL |
<=0 |
0 (ERR) |
Not null terminated |
Any number |
pMultibyteStr = pUTF8Str |
Any size |
0 (ERR) |
Null terminated |
-1 or > 0 |
!=NULL |
>0 |
Converted characters |
Not null terminated |
> 0 |
!=NULL |
>0 |
Converted characters |
Null terminated |
-1 or > 0 |
!=NULL |
>0 |
Converted characters |
Null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Not null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Error Handling
If pMultiByteStr and pUTF8Str are the same, the macro fails and the error buffer returns PIN_ERR_BAD_ARG. If the macro encounters an invalid character in the source string, the macro fails; it sets the return value to 0 and the error buffer to PIN_ERR_BAD_UTF8 as shown in Table 57-8:
Table 57-8 Error Handling Codes
Returned Error Code | Value | Reserved Bit in ebuf | Error Description |
---|---|---|---|
PIN_ERR_BAD_ARG |
4 |
0 |
Bad argument |
PIN_ERR_BAD_LOCALE |
71 |
0 |
Invalid locale string |
PIN_ERR_CONV_MULTIBYTE |
72 |
1 |
Error in UTF8 to multibyte conversion |
PIN_ERR_BAD_UTF8 |
75 |
0 |
Invalid UTF8 characters |
PIN_ERR_NULL_PTR |
39 |
0 |
Empty string passed |
PIN_ERR_NO_MEM |
1 |
1 |
Can't allocate enough memory for conversion |
PIN_CONVERT_UTF8_TO_STR
This macro calls "PIN_CONVERT_UTF8_TO_MBCS" when _MBCS is defined and calls "PIN_CONVERT_UTF8_TO_UNICODE" when _UNICODE is defined. This macro is called whenever translatable data is retrieved from the database.
Syntax
int32 PIN_CONVERT_UTF8_TO_STR( char *pLocaleStr, char *pUTF8Str, int32 nUTF8Len, unsigned char *pBuffer, int32 nBuffersize, pin_errbuf_t *ebufp);
PIN_CONVERT_UTF8_TO_UNICODE
This macro converts a UTF8 character string to a Unicode string.
Syntax
Note:
You need to use setlocale before calling this macro.
int32 PIN_CONVERT_UTF8_TO_UNICODE( unsigned char *pUTF8Str, int nUTF8, wchar_t *pUnicodeStr, int nUnicode, pin_errbuf_t *ebufp);
Parameters
- pUTF8Str
-
Points to the character string to be converted.
- nUTF8
-
Specifies the number of bytes to be converted in the string pointed to by pUTF8Str.
- pUnicodeStr
-
Points to a buffer that receives the converted Unicode string.
- nUnicode
-
Specifies the size of the buffer pointed to by pUnicodeStr.
- ebufp
-
A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.
Return Values
Table 57-9 lists the return values for PIN_CONVERT_UTF8_TO_UNICODE.
Table 57-9 Values Returned by PIN_CONVERT_UTF8_TO_UNICODE
Source String pUTF8str | Number of Bytes to Convert in Input String nUTF8 | Buffer pUnicodeSrt | Buffer Size in Bytes nUnicode | Returned Value |
---|---|---|---|---|
Null terminated |
Any number |
pUTF8str = pUnicodeSrt |
Any size |
0 (ERR) |
NULL |
Any number |
Any |
Any size |
0 (ERR) |
Any |
< -1 |
Any |
Any size |
0 (ERR) |
Any |
0 |
Any |
Any size |
0 (ERR) |
Null terminated |
-1 or > 0 |
NULL |
!=0 |
0 (ERR) |
Not null terminated |
>0 |
NULL |
!=0 |
0 (ERR) |
Any |
Any number |
!=NULL |
<=0 |
0 (ERR) |
Not null terminated |
Any number |
pUTF8str = pUnicodeSrt |
Any size |
0 (ERR) |
Null terminated |
-1 or > 0 |
!=NULL |
>0 |
Converted characters |
Not null terminated |
> 0 |
!=NULL |
>0 |
Converted characters |
Null terminated |
-1 or > 0 |
!=NULL |
>0 |
Converted characters |
Null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Not null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Null terminated |
-1 or > 0 |
NULL |
0 |
Required buffer size |
Error Handling
If pUnicodeStr and pUTF8Str are the same, the macro fails and the error buffer returns PIN_ERR_BAD_ARG. If the macro encounters an invalid character in the source string, the macro fails; it sets the return value to 0 and the error buffer to PIN_ERR_BAD_UTF8 as shown in Table 57-10:
Table 57-10 Error Handling Codes
Returned Error Code | Value | Reserved Bit in ebuf | Error Description |
---|---|---|---|
PIN_ERR_BAD_ARG |
4 |
0 |
Bad argument |
PIN_ERR_CONV_UNICODE |
73 |
1 |
Error in UTF8 to Unicode conversion |
PIN_ERR_BAD_UTF8 |
75 |
0 |
Invalid UTF8 characters |
pin_IsValidUtf8
This function determines whether a specified string is using UTF8 encoding.
Syntax
int32 pin_IsValidUTF8( unsigned char *pUTF8Str, int32 nUTF8Len, pin_errbuf_t *ebufp);
Parameters
- pUTF8Str
-
Points to the character string to be checked for UTF8 encoding.
- nUTF8Len
-
Specifies the number of bytes to be checked in the string pointed to by pUTF8Str.
- ebufp
-
A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.
Return Values
Returns a positive value if pUTF8Str is a valid UTF8 string.
Error Handling
This macro returns 0 if pUTF8Str is not a valid UTF8 string or if any errors occur. Table 57-11 lists the error codes.
Table 57-11 Error Handling Codes
Returned Error Code | Value | Reserved Bit in ebuf | Error Description |
---|---|---|---|
PIN_ERR_NULL_PTR |
39 |
0 |
Empty string passed |
PIN_ERR_BAD_UTF8 |
75 |
0 |
Invalid UTF8 characters |
PIN_MBSLEN
This macro determines the length of the multibyte string.
Syntax
int32 PIN_MBSLEN( char *pLocaleStr, char *pMultiByteStr, pin_errbuf_t *ebufp);
Parameters
- pLocaleStr
-
Indicates the locale information of the input multibyte string. The locale string argument can have the following values:
-
en_US - US-English locale
-
"" - System default locale
-
NULL - Where LC_CTYPE is set to the appropriate locale before calling the macro
-
- pMultiByteStr
-
Points to the multibyte character string.
- ebufp
-
A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.
Return Values
Returns the length of the multibyte string.
Error Handling
This macro returns 0 if any errors occur.
PIN_SETLOCALE
This macro sets, changes, or queries some or all of the current program locale, specified by locale and category. Locale-dependent categories include date and currency formats.
Syntax
char* PIN_SETLOCALE( const int n_category, char *locale_p, pin_errbuf_t *ebufp); *int32 PIN_MBSLEN( const int ncategory, char *locale_p, pin_errbuf_t *ebufp);
Parameters
- n_category
-
The parts of a program's locale that are affected. The macros used for category and the parts of the program they affect are:
-
LC_ALL – All categories, as listed below.
-
LC_COLLATE – The strcoll, _stricoll, wcscoll, _wcsicoll, and strxfrm macros.
-
LC_CTYPE – The character-handling functions (except isdigit, isxdigit, mbstowcs, and mbtowc, which are unaffected).
-
LC_MONETARY – Monetary format information returned by the localeconv function.
-
LC_NUMERIC – Decimal-point character for the formatted output routines (such as printf), for the data-conversion routines, and for the noncurrency formatting information returned by localeconv.
-
LC_TIME – The strftime and wcsftime functions.
-
Return Values
Returns a pointer to the string associated with the specified locale and category.
Error Handling
If the locale or category is invalid, the macro returns a null pointer and sets the error buffer to PIN_ERR_BAD_LOCALE as shown in Table 57-12:
Table 57-12 Error Handling Code
Returned Error Code | Value | Reserved Bit in ebuf | Error Description |
---|---|---|---|
PIN_ERR_BAD_LOCALE |
71 |
0 |
Invalid locale string |
Conversion Code Example
The following code sample shows how to get the locale from the account information and convert the locale from BRM locale to platform locale:
vp = PIN_FLIST_FLD_GET(tmp_acctinfo_flistp, PIN_FLD_LOCALE, 1, ebufp); if (vp == NULL) { /* Set default locale */ strcpy(infranet_locale, "en_US"); } else { strcpy(infranet_locale, (char *)vp); } locale = PIN_MAP_INFRANET_TO_PLATFORM_LOCALE(infranet_locale, ebufp);
The following function shows how to convert a UTF8 string to an MBCS string:
static char * fm_inv_pol_convert_utf8_to_str( char *orig_str, char *locale, pin_errbuf_t *ebufp) { int orig_size = 0; int dest_size = 0; char *strbuf = NULL; orig_size = strlen((char *)orig_str) + 1; /* First round, get the required buffer size for output string */ dest_size = PIN_CONVERT_UTF8_TO_MBCS(locale, (unsigned char *)orig_str, orig_size, NULL, 0, ebufp); if (dest_size == 0) { if( PIN_ERR_IS_ERR( ebufp )) { PIN_ERR_LOG_EBUF(PIN_ERR_LEVEL_DEBUG, "PIN_CONVERT_UTF8_TO_MBCS Failed", ebufp); PIN_ERR_CLEAR_ERR(ebufp); } return NULL; } strbuf = (char *)pin_malloc(sizeof(char)*(dest_size + 1)); if (strbuf == NULL) { pin_set_err(ebufp, PIN_ERRLOC_FM, PIN_ERRCLASS_SYSTEM_DETERMINATE, PIN_ERR_NO_MEM, 0, 0, 0); PIN_ERR_LOG_EBUF(PIN_ERR_LEVEL_DEBUG, "PIN_CONVERT_UTF8_TO_MBCS Failed", ebufp); PIN_ERR_CLEAR_ERR(ebufp); return NULL; } /* Second round, do the string conversion */ dest_size = PIN_CONVERT_UTF8_TO_MBCS(locale, (unsigned char *)orig_str, orig_size, strbuf, dest_size + 1, ebufp); if (dest_size == 0) { if( PIN_ERR_IS_ERR( ebufp )) { PIN_ERR_LOG_EBUF(PIN_ERR_LEVEL_DEBUG, "PIN_CONVERT_UTF8_TO_MBCS Failed", ebufp); PIN_ERR_CLEAR_ERR(ebufp); } pin_free(strbuf); return NULL; } return strbuf; }