Handling Non-ASCII Code on the BRM Server

57 Handling Non-ASCII Code on the BRM Server

Learn how to use character-encoding conversion-layer macros with languages other than English (that is, any non-ASCII character encoding) in your Oracle Communications Billing and Revenue Management (BRM) system.

Topics in this document:

About Character-Encoding Conversion

To work with different languages, BRM applications must use a character encoding that supports them. To support any Western European language or East Asian language, the macros described in this document are required. You must use the conversion macros with any BRM client, server, or Web application localization that is not written in Java. Without these macros, only the 7-bit ASCII encoding works.

Note:

These macros are not required for English language applications, but using the macros facilitates later translations.

BRM supports localizations using Latin 1 and some of the East Asian encodings for Japanese, Korean, Traditional Chinese, and Simplified Chinese only.

About Converting Multibyte or Unicode to and from UTF8

Figure 57-1 shows the relationships between the BRM applications and the character-encoding conversion layer:

Figure 57-1 BRM Applications and the Character-Encoding Conversion Layer

Description of "Figure 57-1 BRM Applications and the Character-Encoding Conversion Layer"

If you are developing a command-line application or a third-party integration, use the macros that convert either Unicode or multibyte input strings to UTF8 and from UTF8:

The macros check the defined preprocessor directive (_MBCS and _UNICODE) to call the direct conversion macros and call the supporting function and macro.

If you are developing a BRM client application or working with multibyte or Unicode only, use the direct conversion macros to change the character encoding. See "About Converting Multibyte or Unicode to and from UTF8":

The macros are located in the Portal Communication Module (PCM) library. The header file is in pcm.h.

Direct Conversion Macros

Table 57-1 lists the functions available in the direct conversion macros.

Table 57-1 Direct Conversion Macros

Function	Description
PIN_CONVERT_MBCS_TO_UTF8	Converts a multibyte character string to UTF8.
PIN_CONVERT_UNICODE_TO_UTF8	Converts Unicode characters to UTF8.
PIN_CONVERT_UTF8_TO_MBCS	Converts UTF8 characters to multibyte.
PIN_CONVERT_UTF8_TO_UNICODE	Converts a UTF8 character string to a Unicode string.

Supporting Functions and Macros

Table 57-2 lists the supporting functions and macros.

Table 57-2 Supporting Functions and Macros

Function	Description
pin_IsValidUtf8	Determines whether a specified string is using UTF8 encoding.
PIN_MBSLEN	Determines the length of the multibyte string.
PIN_SETLOCALE	Sets, changes, or queries some or all of the current program locale, specified by locale and category.

Universal Macros

Table 57-3 lists the universal macros.

Table 57-3 Universal Macros

Function	Description
PIN_CONVERT_STR_TO_UTF8	Converts translatable database data to UTF8.
PIN_CONVERT_UTF8_TO_STR	Calls PIN_CONVERT_UTF8_TO_MBCS when _MBCS is defined, and calls PIN_CONVERT_UTF8_TO_UNICODE when _UNICODE is defined. This macro is called whenever translatable data is retrieved from the database.

PIN_CONVERT_MBCS_TO_UTF8

This macro converts a multibyte character string to UTF8.

Note:

You need to use setlocale before calling this macro.

Syntax

int32 
PIN_CONVERT_MBCS_TO_UTF8(
               char              *pLocaleStr,
               char              *pMultiByteStr, 
               int32              nMultiByteLen, 
               unsigned char     *pUTF8Str, 
               int32              nUTF8size, 
               pin_errbuf_t      *ebufp);

Parameters

pLocaleStr

Indicates the locale of the multibyte string input. The locale string argument can have following values:

en_US - US-English locale
"" - System default locale
NULL - Where LC_CTYPE is set to the appropriate locale before calling the macro

pMultiByteStr

Points to the character string to be converted.

nMultiByteLen

Specifies the number of characters to be converted in the string pointed to by pMultiByteStr. If this value is 1, the string is assumed to be NULL-terminated and the length is calculated automatically.

pUTF8Str

Points to the buffer that receives the converted UTF8 string.

nUTF8size

Specifies the size of the buffer pointed to by pUTF8Str.

ebufp

A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.

Return Values

Table 57-4 lists the values returned by PIN_CONVERT_MBCS_TO_UTF8.

Table 57-4 PIN_CONVERT_MBCS_TO_UTF8 Return Values

SourceString pMultiByteStr	Number of Characters to Convert in Input String nMultiByteLen	Buffer pUTF8Str	Buffer Size in Bytes nUTF8size	Return Value
Null terminated	Any number	pMultiByteStr = pUTF8Str	Any size	0 (ERR)
NULL	Any number	Any	Any size	0 (ERR)
Any	< -1	Any	Any size	0 (ERR)
Any	0	Any	Any size	0 (ERR)
Null terminated	-1 or > 0	NULL	!=0	0 (ERR)
Not null terminated	>0	NULL	!=0	0 (ERR)
Any	Any number	!=NULL	<=0	0 (ERR)
Not null terminated	Any number	pMultiByteStr = pUTF8Str	Any size	0 (ERR)
Null terminated	-1 or > 0	!=NULL	>0	Converted characters
Not null terminated	> 0	!=NULL	>0	Converted characters
Null terminated	-1 or > 0	!=NULL	>0	Converted characters
Null terminated	-1 or > 0	NULL	0	Required buffer size
Not null terminated	-1 or > 0	NULL	0	Required buffer size
Null terminated	-1 or > 0	NULL	0	Required buffer size

Error Handling

If pMultiByteStr and pUTF8Str are the same, the macro fails and the error buffer returns PIN_ERR_BAD_ARG. If the macro encounters an invalid character in the source string, the macro fails; it sets the return value to 0 and the error buffer to the respective error code as shown in Table 57-5:

Table 57-5 Error Codes

Returned Error Code	Value	Error Description
PIN_ERR_BAD_ARG	4	Bad argument
PIN_ERR_BAD_LOCALE	71	Invalid locale string
PIN_ERR_CONV_MULTIBYTE	72	Error in multibyte to UTF8 conversion
PIN_ERR_NULL_PTR	39	Empty string passed

PIN_CONVERT_STR_TO_UTF8

This macro converts translatable database data to UTF8.

Syntax

int32 
PIN_CONVERT_STR_TO_UTF8(
               char              *pLocaleStr, 
               char              *pSourceStr, 
               int32              nSourceLen, 
               unsigned char     *pUTF8Str, 
               int32              nUTF8size, 
               pin_errbuf_t      *ebufp);

PIN_CONVERT_UNICODE_TO_UTF8

This macro converts Unicode characters to UTF8.

Note:

You need to use setlocale before calling this macro.

Syntax

int32 
PIN_CONVERT_UNICODE_TO_UTF8( 
               wchar_t           *pUnicodeStr,
               int                nUnicodeLen, 
               unsigned char     *pUTF8Str, 
               int                nUTF8, 
               pin_errbuf_t      *ebufp);

Parameters

pUnicodeStr: Points to the character string to be converted.
nUnicode: Specifies the number of characters to be translated in the string pointed to by pUnicodeStr.
pUTF8Str: Points to a buffer that receives the translated UTF8 string.
nUTF8: Specifies the size of the buffer pointed to by pUTF8Str.

Return Values

Returns NULL in the error buffer if the macro is successful. Returns the cause of the error if the macro fails.

Error Handling

If pUnicodeStr and pUTF8Str are the same, the macro fails, and the error buffer returns PIN_ERR_BAD_ARG. If the macro encounters an invalid character in the source string, the macro fails; it sets nUTF8 to 0 and sets the error buffer to the respective error code as shown in Table 57-6:

Table 57-6 Error Handling Codes

Returned Error Code	Value	Reserved Bit in ebuf	Error Description
PIN_ERR_BAD_ARG	4	0	Bad argument
PIN_ERR_CONV_UNICODE	73	1	Error in UTF8 to Unicode conversion
PIN_ERR_BAD_UTF8	75	0	Invalid UTF8 characters
PIN_ERR_NULL_PTR	39	0	Empty string passed

PIN_CONVERT_UTF8_TO_MBCS

This macro converts UTF8 characters to multibyte.

Note:

You need to use setlocale before calling this macro.

Syntax

int32 
PIN_CONVERT_UTF8_TO_MBCS(
               char              *pLocaleStr,
               unsigned char     *pUTF8Str, 
               int                nUTF8Len, 
               char              *pMultiByteStr, 
               int                nMultiByte, 
               pin_errbuf_t      *ebufp);

Parameters

pLocaleStr

Indicates the locale of the multibyte string input. The locale string argument can have following values:

en_US - US-English locale
"" - System default locale
NULL - Where LC_CTYPE is set to the appropriate locale before calling the macro

pUTF8Str

Points to the character string to be converted.

nUTF8Len

Specifies the number of bytes to be converted in the string pointed to by pUTF8Str.

pMultiByteStr

Points to the buffer that receives the converted multibyte string.

nMultiByte

Specifies the size of the buffer pointed to by pMultiByteStr.

ebufp

A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.

Return Values

Table 57-7 lists the values returned by PIN_CONVERT_UTF8_TO_MBCS.

Table 57-7 Values Returned by PIN_CONVERT_UTF8_TO_MBCS

Source String pUTF8Str	Number of Characters to be Converted in Input String nUTF8Len	Buffer pMultibyteStr	Buffer Size in Bytes nMultibyte	Returned Value
Null terminated	Any number	pMultibyteStr = pUTF8Str	Any size	0 (ERR)
NULL	Any number	Any	Any size	0 (ERR)
Any	< -1	Any	Any size	0 (ERR)
Any	0	Any	Any size	0 (ERR)
Null terminated	-1 or > 0	NULL	!=0	0 (ERR)
Not null terminated	>0	NULL	!=0	0 (ERR)
Any	Any number	!=NULL	<=0	0 (ERR)
Not null terminated	Any number	pMultibyteStr = pUTF8Str	Any size	0 (ERR)
Null terminated	-1 or > 0	!=NULL	>0	Converted characters
Not null terminated	> 0	!=NULL	>0	Converted characters
Null terminated	-1 or > 0	!=NULL	>0	Converted characters
Null terminated	-1 or > 0	NULL	0	Required buffer size
Not null terminated	-1 or > 0	NULL	0	Required buffer size
Null terminated	-1 or > 0	NULL	0	Required buffer size

Error Handling

Table 57-8 Error Handling Codes

Returned Error Code	Value	Reserved Bit in ebuf	Error Description
PIN_ERR_BAD_ARG	4	0	Bad argument
PIN_ERR_BAD_LOCALE	71	0	Invalid locale string
PIN_ERR_CONV_MULTIBYTE	72	1	Error in UTF8 to multibyte conversion
PIN_ERR_BAD_UTF8	75	0	Invalid UTF8 characters
PIN_ERR_NULL_PTR	39	0	Empty string passed
PIN_ERR_NO_MEM	1	1	Can't allocate enough memory for conversion

PIN_CONVERT_UTF8_TO_STR

This macro calls "PIN_CONVERT_UTF8_TO_MBCS" when _MBCS is defined and calls "PIN_CONVERT_UTF8_TO_UNICODE" when _UNICODE is defined. This macro is called whenever translatable data is retrieved from the database.

Syntax

int32 
PIN_CONVERT_UTF8_TO_STR(
               char              *pLocaleStr, 
               char              *pUTF8Str, 
               int32              nUTF8Len, 
               unsigned char     *pBuffer, 
               int32              nBuffersize, 
               pin_errbuf_t      *ebufp);

PIN_CONVERT_UTF8_TO_UNICODE

This macro converts a UTF8 character string to a Unicode string.

Syntax

Note:

You need to use setlocale before calling this macro.

int32 
PIN_CONVERT_UTF8_TO_UNICODE(
               unsigned char     *pUTF8Str,
               int                nUTF8,
               wchar_t           *pUnicodeStr,
               int                nUnicode, 
               pin_errbuf_t      *ebufp);

Parameters

pUTF8Str: Points to the character string to be converted.
nUTF8: Specifies the number of bytes to be converted in the string pointed to by pUTF8Str.
pUnicodeStr: Points to a buffer that receives the converted Unicode string.
nUnicode: Specifies the size of the buffer pointed to by pUnicodeStr.
ebufp: A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.

Return Values

Table 57-9 lists the return values for PIN_CONVERT_UTF8_TO_UNICODE.

Table 57-9 Values Returned by PIN_CONVERT_UTF8_TO_UNICODE

Source String pUTF8str	Number of Bytes to Convert in Input String nUTF8	Buffer pUnicodeSrt	Buffer Size in Bytes nUnicode	Returned Value
Null terminated	Any number	pUTF8str = pUnicodeSrt	Any size	0 (ERR)
NULL	Any number	Any	Any size	0 (ERR)
Any	< -1	Any	Any size	0 (ERR)
Any	0	Any	Any size	0 (ERR)
Null terminated	-1 or > 0	NULL	!=0	0 (ERR)
Not null terminated	>0	NULL	!=0	0 (ERR)
Any	Any number	!=NULL	<=0	0 (ERR)
Not null terminated	Any number	pUTF8str = pUnicodeSrt	Any size	0 (ERR)
Null terminated	-1 or > 0	!=NULL	>0	Converted characters
Not null terminated	> 0	!=NULL	>0	Converted characters
Null terminated	-1 or > 0	!=NULL	>0	Converted characters
Null terminated	-1 or > 0	NULL	0	Required buffer size
Not null terminated	-1 or > 0	NULL	0	Required buffer size
Null terminated	-1 or > 0	NULL	0	Required buffer size

Error Handling

If pUnicodeStr and pUTF8Str are the same, the macro fails and the error buffer returns PIN_ERR_BAD_ARG. If the macro encounters an invalid character in the source string, the macro fails; it sets the return value to 0 and the error buffer to PIN_ERR_BAD_UTF8 as shown in Table 57-10:

Table 57-10 Error Handling Codes

Returned Error Code	Value	Reserved Bit in ebuf	Error Description
PIN_ERR_BAD_ARG	4	0	Bad argument
PIN_ERR_CONV_UNICODE	73	1	Error in UTF8 to Unicode conversion
PIN_ERR_BAD_UTF8	75	0	Invalid UTF8 characters

pin_IsValidUtf8

This function determines whether a specified string is using UTF8 encoding.

Syntax

int32 
pin_IsValidUTF8(
               unsigned char     *pUTF8Str, 
               int32              nUTF8Len, 
               pin_errbuf_t      *ebufp);

Parameters

pUTF8Str: Points to the character string to be checked for UTF8 encoding.
nUTF8Len: Specifies the number of bytes to be checked in the string pointed to by pUTF8Str.
ebufp: A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.

Return Values

Returns a positive value if pUTF8Str is a valid UTF8 string.

Error Handling

This macro returns 0 if pUTF8Str is not a valid UTF8 string or if any errors occur. Table 57-11 lists the error codes.

Table 57-11 Error Handling Codes

Returned Error Code	Value	Reserved Bit in ebuf	Error Description
PIN_ERR_NULL_PTR	39	0	Empty string passed
PIN_ERR_BAD_UTF8	75	0	Invalid UTF8 characters

PIN_MBSLEN

This macro determines the length of the multibyte string.

Syntax

int32 
PIN_MBSLEN(
               char             *pLocaleStr,
               char             *pMultiByteStr, 
               pin_errbuf_t     *ebufp);

Parameters

pLocaleStr

Indicates the locale information of the input multibyte string. The locale string argument can have the following values:

en_US - US-English locale
"" - System default locale
NULL - Where LC_CTYPE is set to the appropriate locale before calling the macro

pMultiByteStr

Points to the multibyte character string.

ebufp

A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.

Return Values

Returns the length of the multibyte string.

Error Handling

This macro returns 0 if any errors occur.

PIN_SETLOCALE

This macro sets, changes, or queries some or all of the current program locale, specified by locale and category. Locale-dependent categories include date and currency formats.

Syntax

char*
PIN_SETLOCALE(
               const int         n_category, 
               char             *locale_p, 
               pin_errbuf_t     *ebufp);
  
*int32 
PIN_MBSLEN(
               const int         ncategory,
               char             *locale_p,
               pin_errbuf_t     *ebufp);

Parameters

n_category

The parts of a program's locale that are affected. The macros used for category and the parts of the program they affect are:

LC_ALL – All categories, as listed below.
LC_COLLATE – The strcoll, _stricoll, wcscoll, _wcsicoll, and strxfrm macros.
LC_CTYPE – The character-handling functions (except isdigit, isxdigit, mbstowcs, and mbtowc, which are unaffected).
LC_MONETARY – Monetary format information returned by the localeconv function.
LC_NUMERIC – Decimal-point character for the formatted output routines (such as printf), for the data-conversion routines, and for the noncurrency formatting information returned by localeconv.
LC_TIME – The strftime and wcsftime functions.

locale_p

Indicates the locale.

ebufp

A pointer to the error buffer. If this macro is successful, the error buffer is NULL; otherwise, it indicates the cause of the error.

The null pointer is a special directive that tells PIN_SETLOCALE to query rather than set the international environment.

Return Values

Returns a pointer to the string associated with the specified locale and category.

Error Handling

If the locale or category is invalid, the macro returns a null pointer and sets the error buffer to PIN_ERR_BAD_LOCALE as shown in Table 57-12:

Table 57-12 Error Handling Code

Returned Error Code	Value	Reserved Bit in ebuf	Error Description
PIN_ERR_BAD_LOCALE	71	0	Invalid locale string

Conversion Code Example

The following code sample shows how to get the locale from the account information and convert the locale from BRM locale to platform locale:

vp = PIN_FLIST_FLD_GET(tmp_acctinfo_flistp, PIN_FLD_LOCALE, 
            1, ebufp);
     if (vp == NULL) { /* Set default locale */
            strcpy(infranet_locale, "en_US");
    } else {
            strcpy(infranet_locale, (char *)vp);
    }
    locale = PIN_MAP_INFRANET_TO_PLATFORM_LOCALE(infranet_locale, 
            ebufp);

The following function shows how to convert a UTF8 string to an MBCS string:

static char *
fm_inv_pol_convert_utf8_to_str(
        char                    *orig_str,
        char                    *locale,
        pin_errbuf_t            *ebufp)
{
        int             orig_size = 0;
        int             dest_size = 0;
        char        *strbuf = NULL;
  
        orig_size = strlen((char *)orig_str) + 1;
  
        /* First round, get the required buffer size for output string */
        dest_size = PIN_CONVERT_UTF8_TO_MBCS(locale,
        (unsigned char *)orig_str,
        orig_size, NULL, 0, ebufp);
  
    if (dest_size == 0) {
        if( PIN_ERR_IS_ERR( ebufp )) {
            PIN_ERR_LOG_EBUF(PIN_ERR_LEVEL_DEBUG,
                "PIN_CONVERT_UTF8_TO_MBCS Failed",
                ebufp);
            PIN_ERR_CLEAR_ERR(ebufp);
        }
        return NULL;
    }
  
    strbuf = (char *)pin_malloc(sizeof(char)*(dest_size + 1));
    if (strbuf == NULL) {
        pin_set_err(ebufp, PIN_ERRLOC_FM,
            PIN_ERRCLASS_SYSTEM_DETERMINATE,
            PIN_ERR_NO_MEM, 0, 0, 0);
        PIN_ERR_LOG_EBUF(PIN_ERR_LEVEL_DEBUG,
            "PIN_CONVERT_UTF8_TO_MBCS Failed",
            ebufp);
        PIN_ERR_CLEAR_ERR(ebufp);
        return NULL;
    }
  
    /* Second round, do the string conversion */
    dest_size = PIN_CONVERT_UTF8_TO_MBCS(locale,
        (unsigned char *)orig_str,
        orig_size, strbuf, dest_size + 1, ebufp);
  
    if (dest_size == 0) {
        if( PIN_ERR_IS_ERR( ebufp )) {
            PIN_ERR_LOG_EBUF(PIN_ERR_LEVEL_DEBUG,
                "PIN_CONVERT_UTF8_TO_MBCS Failed",
                ebufp);
            PIN_ERR_CLEAR_ERR(ebufp);
        }
        pin_free(strbuf);
        return NULL;
    }
  
    return strbuf;
}