Creating Log Mappings


	Corporate Info \| News \| Solutions \| Products \| Partners \| Services \| Events \| Download \| How To Buy
	http://www.oracle.com/technology/documentation/index.html \| Search \| PDF Files \| Contact

Log Central Doc Home \| Configuration Guide \| Previous Topic \| Next Topic \| Contents \| Index

The following sections describe how to create log mappings:

What Is a Log Mapping?

Where to Define Log Mappings

Using Multiple Log Mappings

Mapping Options

Date Formats for the -D Option

Specifying Option Values

Mapping Dates

Working with Metacharacters

Specifying Multiple Separators

Working with Field Lengths

Example: Mapping a System Log

What Is a Log Mapping?

A log mapping is a list of options. For example:

-S |! -o %F8 -p sony -b %F12

A Log Monitor uses log mappings to translate log messages into the Log Central format, which is described in Log Central Message and Message Definition Formats. Log Central provides predefined mappings for several types of log files. For descriptions of these predefined mappings, see Predefined Log Mappings. This chapter describes how to create mappings for additional types of log files.

The following table describes how the previous mapping works.

Table 5-1 Mapping Example

Mapping Option

Description

-S |!

Specifies that | and ! are the separator characters between the fields in the incoming message

-o %F8

Tells the Log Monitor to use the incoming message's 8th field as the host name in the resulting message

-p sony

Tells the Log Monitor to forward the message only if "sony" is in the message

-b %F12

Tell the Log Monitor to use the incoming message's 12th field as the body of the resulting message

Mapping Option	Description
-S \|!	Specifies that \| and ! are the separator characters between the fields in the incoming message
-o %F8	Tells the Log Monitor to use the incoming message's 8th field as the host name in the resulting message
-p sony	Tells the Log Monitor to forward the message only if "sony" is in the message
-b %F12	Tell the Log Monitor to use the incoming message's 12th field as the body of the resulting message

Where to Define Log Mappings

If you are going to start the Log Monitors automatically when you start Log Central with the start_messaging command, you need to:

Define the log mappings in the messaging configuration file.
Include LOG_MONITOR statements in the MANAGED_NODE entry in the messaging configuration file.

If you are going to start the Log Monitors manually with the log_monitor command, you need to define the log mappings in the message mapping file or on the log_monitor command line. If you use the log_monitor command line, you can define only one log mapping.

For information about the messaging configuration file, see Configuration Files. For information about the start_messaging and log_monitor commands, see Commands.

Using Multiple Log Mappings

When you define log mappings in a message mapping file or in the messaging configuration file, the Log Monitor tests an incoming log message against each mapping sequentially. To be forwarded to the Central Collector, a log message needs to be selected by only one of the mappings in the file. You can use multiple mappings to handle different types of log messages that come from the same managed resource.

Mapping Options

Most of the mapping options tell the Log Monitor how to map a field in an incoming message to a field in the resulting message. However, you can use the -p and -x options to discard messages. For example, if you use the -p option in a mapping, the Log Monitor forwards a log message only if it contains the specified pattern. If you use the -x option, the Log Monitor discards log messages that contain the specified pattern. The following table describes the mapping options.

Option	Description
-b body	String that contains the body of the message.
-D date %f "format"	Date format to use in the resulting message. See Date Formats for the -D Option and Mapping Dates.
-d msgid	Message ID to use in the resulting message. The default is 1000.
-e entity	Entity name to use in the resulting message. An entity name is the name that one or more Log Monitors use to register with the proc_monitor process. The default is log_monitor. Each Log Monitor on one managed node must have a unique entity name. If you run a Log Monitor as a daemon (with -t 0), this option is not used.
-I processID	Process ID to use in the resulting message. The default is the process ID of log_monitor.
-M log_level	Logging level to use in the resulting message. This is a one-character string with the following possible values: N—Normal V—Verbose D—Debug S—Special
-m subsystem	Subsystem name to use in the resulting message. The default is none.
-n function	Function name to use in the resulting message. The default is none.
-o hostname	Host name to use in the resulting message. The default is the machine on which the Log Monitor is running.
-p pattern	Tells the Log Monitor to forward a log message only if it contains the specified pattern, which can include the metacharacters described in Working with Metacharacters.
-S separators	Separators for parsing field values when the mapping includes %F format symbols. Table 5-4 describes %F. If you specify more than one separator, the Log Monitor uses all of them. For more information, see Specifying Multiple Separators.
-u userID	User ID to use in the resulting message.The default is the current user, which is the owner of the log_monitor process.
-x pattern	Tells the Log Monitor to discard a log message if it contains the specified pattern, which can include the metacharacters described in Working with Metacharacters. The message can still be forwarded if it satisfies another mapping in the message mapping file or messaging configuration file.

Date Formats for the -D Option

The following table describes the date formats for the -D option.

Format	Description
%%	A literal percent sign
%a	Abbreviated weekday name (for example, Sun)
%A	Full weekday name (for example, Sunday)
%b	Abbreviated month name (for example, Jan)
%B	Full month name (for example, January)
%d	Day of month (1-31; leading zeroes are permitted but not required)
%D	Date as %m/%d/%y
%h	Same as %b
%H	Hour (0-23; leading zeroes are permitted but not required)
%I	Hour (0-12; leading zeroes are permitted but not required)
%j	Day number of year (001-366; leading zeroes are permitted but not required)
%m	Month number (1-12; leading zeroes are permitted but not required)
%M	Minute (0-59; leading zeroes are permitted but not required)
%p	Local equivalent of AM or PM
%r	Time as %I:%M:%S %p
%S	Seconds (0-59; leading zeroes are permitted but not required)
%T	Time as %H:%M:%S
%y	Year in the century (0-99; leading zeroes are permitted but not required)
%Y	Year, including the century (for example, 2000)

Specifying Option Values

An option value can be a literal or a format. You can mix literals and formats in the same option value. To use a literal, follow the option with the specific value. For example:

-b "This is the message body."
-m "Major function"

To use a format, use format symbols to extract the value from the incoming log message. For example:

-o %F8

To use a literal and a format, precede the literal with %V, which is a format symbol. For example:

-o %F11%Vmachine

The following table describes the format symbols.

Table 5-4 Format Symbols

Format Symbol

Description

Examples

%C

Starting character position for the value. Must be followed by L or S to terminate the value. L specifies the number of characters in the value. S specifies that the following character is a separator.

-u %C10L4

Selects the 10th, 11th, 12th, and 13th characters from the incoming message to use as the user ID in the resulting message.

-u %C10S|

Selects the string that starts at the 10th character and ends immediately before the next | character to use as the user ID in the resulting message.

%F

Used with an integer to specify a field. Must be accompanied by the -S option. Fields are numbered starting with 1 (not 0).

-m %F1 -S |

Selects the first field from the incoming message and uses it as the subsystem name in the resulting message. The field separator is the vertical bar.

%f

Format for the date value. This option is used with the -D option only. Date Formats for the -D Option describes the date formats.

-D %F3%f%T

Selects the third field, which is in the %T format, to use as the date in the resulting message.

-D "%F1%V %F2%V %F3%f%h %d %T" -S " "

Selects the first field, which is in the %h format; the second field, which is in the %d format; and the third field, which is in the %T format, to use as the date in the resulting message. You need to use double quotes if the option is on the command line or contains one or more embedded spaces.

There must be separators between the date format specifiers with %f. The same separator should appear in the value specified for -D. This is why %V is used in the first part of the specification to fill in the corresponding separators between the fields generated by Log Monitor.

%V

User-defined value.

-n %C3L5%F11%V"minor function" -S :

Selects the five-character string starting at the 3rd character in the 11th field and adds the string "minor function" to use as the function name in the resulting message. The field separator is the colon.

Format Symbol	Description	Examples
%C	Starting character position for the value. Must be followed by L or S to terminate the value. L specifies the number of characters in the value. S specifies that the following character is a separator.	-u %C10L4 Selects the 10th, 11th, 12th, and 13th characters from the incoming message to use as the user ID in the resulting message. -u %C10S\| Selects the string that starts at the 10th character and ends immediately before the next \| character to use as the user ID in the resulting message.
%F	Used with an integer to specify a field. Must be accompanied by the -S option. Fields are numbered starting with 1 (not 0).	-m %F1 -S \| Selects the first field from the incoming message and uses it as the subsystem name in the resulting message. The field separator is the vertical bar.
%f	Format for the date value. This option is used with the -D option only. Date Formats for the -D Option describes the date formats.	-D %F3%f%T Selects the third field, which is in the %T format, to use as the date in the resulting message. -D "%F1%V %F2%V %F3%f%h %d %T" -S " " Selects the first field, which is in the %h format; the second field, which is in the %d format; and the third field, which is in the %T format, to use as the date in the resulting message. You need to use double quotes if the option is on the command line or contains one or more embedded spaces. There must be separators between the date format specifiers with %f. The same separator should appear in the value specified for -D. This is why %V is used in the first part of the specification to fill in the corresponding separators between the fields generated by Log Monitor.
%V	User-defined value.	-n %C3L5%F11%V"minor function" -S : Selects the five-character string starting at the 3rd character in the 11th field and adds the string "minor function" to use as the function name in the resulting message. The field separator is the colon.

Mapping Dates

The following rules apply for converting dates from date and time values in an incoming log message to date and time values in the Log Central log message:

If only the weekday is in the incoming message, without specifying a day of the month, today is assumed if the incoming weekday is equal to the current weekday. Otherwise, the corresponding day from the prevous six days is assumed.
If only the month is in the incoming message, without specifying a year, the current month is assumed if the incoming month is equal to the current month. Otherwise, the corresponding month from the previous 11 months is assumed. The first day of the month is assumed if no day is provided.
If only the time is in the incoming message, without specifying a date, today is assumed. If no hour, minute, and second are provided, the current hour, minute, and second are assumed.

The following example illustrates these rules for a current date of Tuesday September 19 12:19:47 PDT 2000.

Listing 5-1 Example for Mapping Dates

Input            Line in Template      Date
______________________________________________________________
Tue              %a                    Sep 19 12:19:47 PDT 2000
Mon              %a                    Sep 18 12:19:47 PDT 2000
Fri              %a                    Sep 15 12:19:47 PDT 2000
September        %B                    Sep 1 12:19:47 PDT 2000
January          %B                    Jan 1 12:19:47 PDT 2000
December         %B                    Dec 1 12:19:47 PDT 1999
Sep Tue          %b %a                 Sep 19 12:19:47 PDT 2000
Jan Sat          %b %a                 Jan 1 12:19:47 PDT 2000
Dec Tue          %b %a                 Dec 7 12:19:47 PDT 1999
Jan Fri 2003     %b %a %Y              Jan 3 12:19:47 PDT 2003
Fri 9            %a %H                 Sep 15 09:00:00 PDT 2000
Feb 10:30        %b %H:%S              Feb 1 10:00:30 PDT 2000
10:30            %H:%M                 Sep 19 10:30:00 PDT 2000
13:30            %H:%M                 Sep 19 13:30:00 PDT 2000

Working with Metacharacters

With the -p and -x options, you can use metacharacters to select a range of values. For example, to specify a value from 97 to 99, you can use 9[7-9]. The following table lists the metacharacters you can use with the -p and -x options.

Table 5-5 Metacharacters

Expression

Description

?

Matches any single character except a newline character.

%

Matches the beginning of the line. For example, %abc matches a string only if the letters abc are the first three characters of a line. The % symbol does not have its special metacharacter role if it is not at the beginning of a line.

$

Matches the end of a line. For example, xyz$ matches a string only if the letters xyz are the last three characters on the line. The $ symbol does not have its special metacharacter role if it is not at the end of a line.

@c

Escapes the character that follows the @. When followed by any metacharacter, the expression matches the metacharacter itself. For example, @% matches a percent sign, which otherwise would be interpreted as part of an expression that starts at the beginning of a line.

*

Indicates zero or more occurrences of the preceding character or expression. A single character followed by an asterisk is a regular expression that matches zero or more occurrences of that one character. If the expression has multiple matches, it chooses the longest leftmost string that permits a match. For example, in a line starting aaabaa, the expression a* would match aaa. For another example, [a-zA-Z][a-zA-Z]*$ matches lines that end in words. The specification is to match an entire word, that is, one containing one or more alphabetic characters.

[string]

Indicates a string. A nonempty string enclosed in square brackets matches any one character in the string. If the first character is a caret (^), the regular expression matches any character except a newline character and the remaining characters in the string. For example, [^a-zA-Z0-9] matches any nonalphanumeric character. Use a hyphen to indicate a range of consecutive ASCII characters, such as [0-9].

Expression	Description
?	Matches any single character except a newline character.
%	Matches the beginning of the line. For example, %abc matches a string only if the letters abc are the first three characters of a line. The % symbol does not have its special metacharacter role if it is not at the beginning of a line.
$	Matches the end of a line. For example, xyz$ matches a string only if the letters xyz are the last three characters on the line. The $ symbol does not have its special metacharacter role if it is not at the end of a line.
@c	Escapes the character that follows the @. When followed by any metacharacter, the expression matches the metacharacter itself. For example, @% matches a percent sign, which otherwise would be interpreted as part of an expression that starts at the beginning of a line.
*	Indicates zero or more occurrences of the preceding character or expression. A single character followed by an asterisk is a regular expression that matches zero or more occurrences of that one character. If the expression has multiple matches, it chooses the longest leftmost string that permits a match. For example, in a line starting aaabaa, the expression a* would match aaa. For another example, [a-zA-Z][a-zA-Z]*$ matches lines that end in words. The specification is to match an entire word, that is, one containing one or more alphabetic characters.
[string]	Indicates a string. A nonempty string enclosed in square brackets matches any one character in the string. If the first character is a caret (^), the regular expression matches any character except a newline character and the remaining characters in the string. For example, [^a-zA-Z0-9] matches any nonalphanumeric character. Use a hyphen to indicate a range of consecutive ASCII characters, such as [0-9].

Specifying Multiple Separators

When you use the -S option, you can specify multiple separator characters. For an incoming message of abcd^xys^b|bbbb^, Table 5-6 shows how the Log Monitor parses the message with -S ^|, and Table 5-7 shows how the Log Monitor parses the message with -S ^.

Table 5-6 Multiple Separators Example 1: -S ^|

Field Number

Contents

1

abcd

2

xys

3

b

4

bbbb

Field Number	Contents
1	abcd
2	xys
3	b
4	bbbb

Table 5-7 Multiple Separators Example 2: -S ^

Field Number

Contents

1

abcd

2

xys

3

b|bbbb

Field Number	Contents
1	abcd
2	xys
3	b\|bbbb

The Log Monitor ignores separators at the beginning of a message. For example, the fields would be exactly the same if the incoming message was |abcd^xys^b|bbbb^.

Working with Field Lengths

If the length of a string-valued field in an incoming message exceeds its maximum, the Log Monitor truncates the value. For example, if an incoming message contains the user ID Administrator, it would be truncated to Administ. The following table lists the maximum field lengths.

Table 5-8 Maximum Field Lengths

Field

Maximum Length

Subsystem Name

8

User ID

8

Hostname

20

Function name

40

Entity

21

Timestamp

20

Message Body

2000

Field	Maximum Length
Subsystem Name	8
User ID	8
Hostname	20
Function name	40
Entity	21
Timestamp	20
Message Body	2000

Example: Mapping a System Log

This example filters a system log. The following listing contains messages from the UNIX system log (/var/log/syslog).

Listing 5-2 UNIX System Log

May 15 11:06:02 eclipse vmunix: psig: "EM_client" signal 15 was
masked, put back.

May 16 13:51:11 eclipse lpd[8951]: /usr/spool/lpd/lpd-log: No such
file or directory

May 17 10:38:12 eclipse su: 'su webuild' failed for emilie on
/dev/ttyp4

May 17 13:54:28 eclipse vmunix: NFS write error: on host iseult
remote file system full

May 17 13:54:37 eclipse last message repeated 13 times

May 17 14:40:42 eclipse lpd[9290]: /usr/spool/lpd/lpd-log: No such
file or directory

May 17 17:08:09 eclipse su: 'su root' succeeded for emilie on
/dev/ttyp0

The following listing contains a message mapping file.

Listing 5-3 Message Mapping File

-M LM_VERBOSE -D "%F1%V %F2%V %F3%f%h %d %T" -S " " -m NFS -d 123
-o %F4 -p "write error" -u emilie -n %F5 -b %F6-

-D "%F1%V %F2%V %F3%f%h %d %T" -S " " -m AUTH -d 124 -o %F4 -p su:
-u emilie -n %F5 -b %F6-

-D "%F1%V %F2%V %F3%f%h %d %T" -S " " -m PRINT -d 125 -o %F4 -p lpd
-u emilie -n %F5 -b %F6-

The following listing shows the result of the mapping.

Listing 5-4 Result of Mapping

|N|May 16 13:51:11 2000|PRINT|125|eclipse|11593|emilie|lpd[8951]:
|0|1!/usr/spool/lpd/lpd-log: No such file or directory

|N|May 17 10:38:12 2000|AUTH|124|eclipse|11593|emilie|su:
|0|1!'su webuild' failed for emilie on /dev/ttyp4

|V|May 17 13:54:28 2000|NFS|123|eclipse|11593|emilie|vmunix:
|0|1!NFS write error: on host iseult remote file system full

|N|May 17 14:40:42 2000|PRINT|125|eclipse|11593|emilie|lpd[9290]:
|0|1!/usr/spool/lpd/lpd-log: No such file or directory

|N|May 17 17:08:09 2000|AUTH|124|eclipse|11593|emilie|su:
|0|1!'su root' succeeded for emilie on /dev/ttyp0

The following table explains how mapping the third log message in the UNIX system log (Listing 5-2) with the second line in the message mapping file (Listing 5-3) produces the second message in the results (Listing 5-4).

Table 5-9 Mapping Example

Mapping

Input

Meaning

Output

-D "%F1%V %F2%V %F3%f%h %d %T" -S " "

May 17 10:38:12

Time stamp: constructed from fields 1, 2, and 3.

May 17 10:38:12 2000

-m AUTH

Field not present in input.

Subsystem: specified by literal value.

AUTH

-d 124

Field not present in input.

Message ID: specified by literal value.

124

-o %F4

eclipse

Host name: taken from field 4.

eclipse

-p su:

su:

Pattern match required to forward the message.

The message can be forwarded.

-u emilie

Field not present in input.

User name: specified by literal value.

emilie

-n %F5

su:

Function name: taken from field 5.

su:

-b %F6-

'su webuild' failed for emilie on /dev/ttyp4

Message body: constructed from field 6 to the end of line.

'su webuild' failed for emilie on /dev/ttyp4

None.

None.

Reporting mode: the default, LM_NORMAL, which produces an N.

N

None.

None.

Process ID: the default, which is the PID of log_monitor.

11593

None.

None.

Entity: the default , which is 0.

0

None.

None.

Reserved.

1!

Mapping	Input	Meaning	Output
-D "%F1%V %F2%V %F3%f%h %d %T" -S " "	May 17 10:38:12	Time stamp: constructed from fields 1, 2, and 3.	May 17 10:38:12 2000
-m AUTH	Field not present in input.	Subsystem: specified by literal value.	AUTH
-d 124	Field not present in input.	Message ID: specified by literal value.	124
-o %F4	eclipse	Host name: taken from field 4.	eclipse
-p su:	su:	Pattern match required to forward the message.	The message can be forwarded.
-u emilie	Field not present in input.	User name: specified by literal value.	emilie
-n %F5	su:	Function name: taken from field 5.	su:
-b %F6-	'su webuild' failed for emilie on /dev/ttyp4	Message body: constructed from field 6 to the end of line.	'su webuild' failed for emilie on /dev/ttyp4
None.	None.	Reporting mode: the default, LM_NORMAL, which produces an N.	N
None.	None.	Process ID: the default, which is the PID of log_monitor.	11593
None.	None.	Entity: the default , which is 0.	0
None.	None.	Reserved.	1!