Chapter 4. Integrating Logs into Log Central

The Log Monitor program integrates logs into Log Central. Log Monitor does the following:

Monitors application logs for incoming messages.
Selects which incoming log messages are to be forwarded to the Log Central database.
Maps the attributes in log messages into the internal Log Central log message format.

A number of predefined mappings of log formats are provided with Log Monitor. For information on how to start Log Monitor using a predefined mapping, refer to Chapter 8, "Starting and Stopping Log Central." This chapter describes how to construct mappings for additional log files that you want to monitor. Consult Appendix A, "Message Format," for information about the Log Central log message format.

There are two ways to pass a mapping to Log Monitor when it is started:

As a series of options in a Log Monitor configuration file
As options passed on the command line

This chapter describes the configuration file options. Passing the options on the command line, and the syntax used in starting Log Monitor, are discussed in Chapter 8, "Starting and Stopping Log Central."

Log Monitor Configuration File Options

A Log Monitor configuration file is simply a list of up to 20 different Log Monitor filters, each on a separate line. Each filter consists of a list of Log Monitor options.

Filters can be used to specify which messages to drop. For example, if you use the -p pattern option in a filter, a log message that satisfy this filter is then forwarded by Log Monitor. If this is the only filter in the file, any message not matching the pattern specified by the -p option will be discarded.

Alternatively, you can use the -x option to specify particular messages that you do not want selected. To select which message to ignore, the -x option also uses a pattern to determine a match. Using patterns with the -x and -p options is discussed below under "Using the -p and -x Options."

Log Monitor tests each incoming message against each filter sequentially. A log message that fails to be selected by one filter in the configuration file may be selected by one of the other filters. A log message is forwarded by the Log Monitor to the Log Central database if it is selected by at least one of the filters in the configuration file.

You may want to use multiple filters to provide different mappings for different message types.

If you use a configuration file to instruct Log Monitor how to map incoming log messages, the name of the configuration file is passed to Log Monitor in the -f option when it is started.

Table 4-1 summarizes the options that are available for defining a mapping in a configuration file.

**Table 4-1 Log Monitor Configuration File Options**
Argument	Description
`-b` `body`	`body` is a string that contains the body of the message.
`-D` `date` `%f"``format``"`	This option specifies the date format to use in the message. See "Specifying the Date Format (%f)."
`-d` `msgid`	This option specifies the message ID to use in the message. The default is `1000`.
`-I` `processID`	This option specifies the process ID to use in the message. The default is the process ID of the Log Monitor daemon process.
`-M` `log_level`	This option specifies the logging level to use in the message. This is a one-character string. The allowed values and their interpretation are as follows: `N`-Normal `V`-Verbose `D`-Debug `S`-Special
`-m` `subsystem`	This option specifies the module subsystem name to use in the message. The default is `none`.
`-n` `function`	This option specifies the function name to use in the message. The default is `none`.
`-o` `hostname`	This option specifies the host name to use in the message. The default is the machine on which Log Monitor is running.
`-p` `pattern`	This option instructs the program to forward only messages that match `pattern`, which may use special metacharacters. Use the metacharacters defined under "Using the -p and -x Options."
`-S`	This option specifies one or more separators to be used to calculate field values in the input message file for corresponding `%F` specifiers on a command line. (The `%F` format symbol is described in Table 4-2.) If more than one separator is specified, all are used to count the fields. If a message starts with a separator, the text between the first and the second separator is counted as field number 1. Fields are numbered starting with 1 (not 0). If a message does not start with a separator, the first field consists of the text up to the first separator. For examples, see "Multiple Separators with the -S Option."
`-T` `transactionID`	This option specifies the transaction ID to use in the message. The default is 0.
`-u` `userID`	This option specifies the user ID to use in the message. The default is the current user (that is, the owner of the `log_monitor` process).
`-x` `pattern`	This option instructs the program to not forward messages that match `pattern`. To define a pattern, you can use the metacharacters defined under "Using the -p and -x Options."

**Table 4-2 Format Symbols**
Character	Description	Examples
`%F`	Used with an integer to specify a particular field. Must be accompanied by `-S` (from Table 4-1). The `-S` option specifies the character used to separate fields in the message.	`-m %F1 -S '\|'` This selects the first field from the incoming message and uses it as the subsystem name in the corresponding Log Central message to be generated. The field separator is the vertical bar.
`%C`	Specifies the starting character position for the value. Must be followed by the `L` or `S` symbol (not both) to terminate the value. The `L` symbol specifies an absolute length in characters of the value. The `S` symbol specifies that the following character is to be a separator.	`-u %C10L4` This selects the 10th, 11th, 12th, and 13th characters from the incoming message to use as the user ID in the corresponding Log Monitor message to be generated. `-u %C10S\|` This selects the string that starts at the 10th character and ends immediately before the next `\|` character.
`%V`	Specifies user-defined values within formats.	`-n %C3L5%F11%V"This option indicates a MINOR FUNCTION" -S:` This creates a function name entry composed of the five-character string starting at character 3, field 11, and the string `This option indicates a MINOR FUNCTION`. The field separator is the colon.
`%f`	Specifies the format for the date value. This option is used with the `-D` option only. All values of `%f` are detailed in Table 4-4.	`-D %F1%f"``format``"` This specifies that the date value in field 1 is in the format as specified by the format string (`format`). Double quotes are required only if this option is being given on the command line (or if `format` contains one or more embedded spaces).

Specifying Option Values

You can specify how the value for an option is determined in two ways: by literal value and by format. The methods can be mixed.

To specify the option by literal value, follow the option with the specific value. For example:

-u KONG
-b "This is the message body."
-m "Major function"
-d 1234

If you specify both a literal value and a format, the literal must start with the characters %V, which is explained in Table 4-2.

To specify the value by format, use format symbols to extract the value from the application log message. Examples appear in Table 4-2 under %C and %V.

Example of Using Log Monitor with a Configuration File

The following is an example of a command used to start Log Monitor with options in a configuration file.

log_monitor -f forward_options -i /home/demo/demo.mul_fld -t 0

The configuration file is named forward_options. The contents of forward_options are:

-S |! -o %F8 -p sony -b %F12 -T %F10
-S |! -I %F6 -u %F7 -b %F11 -x error
-S |! -m %F3%V=%C30S| -n %F8%F10 -b %F11 -D %F2

Each line pertains to one filter or mapping for the/home/demo/demo.mul_fld log file. Log Monitor works on these filters sequentially. Log Monitor picks one message, applies the three filters one after the other, then moves to the next message in the input file.

The separator specification following the -S option indicates that either | or ! is considered to be a separator. Whichever one the parser encounters signals the start of a new field. For an explanation of the -S option, refer to "Multiple Separators with the -S Option."

Multiple Separators with the -S Option

When you use the -S option, you can specify multiple separator characters. The following example shows how fields are numbered in such situations.

If the following is the incoming message:

abcd^xys^b|bbbb^

Specifying -S^| for the separator results in the following.


Field Number	Contents
1	`abcd`
2	`xys`
3	`b`
4	`bbbb`

Specifying -S^ for the separator results in the following.


Field Number	Contents
1	`abcd`
2	`xys`
3	`b\|bbbb`

Separators at the beginning of a message are ignored. For example, the fields would be exactly the same as the previous table if the incoming message were:

|abcd^xys^b|bbbb^

Using the -p and -x Options

The -p option specifies a pattern that is used to select messages for forwarding. If a message does not match the pattern, it is not selected for forwarding by that filter.

The pattern may simply be a string. For example, if you use:

-p su:

then any message in which su: occurs will be selected. You can also use metacharacters to select a range of values. For example, if you want to select messages with a year value from 97 to 99, you could use:

9[7-9]

to specify such a range.

The -x option specifies a pattern that is used to select messages to be discarded. If a message matches the pattern, it is not selected by that filter. (However, that message might still be selected for forwarding by another filter if you are using multiple filters in the same configuration file.)

Table 4-3 lists the metacharacters you can use with the -p and -x options.

**Table 4-3 Metacharacters**
Expression	Description
`?`	Matches any single character except a newline character.
`%`	Matches the beginning of the line. For example, `%abc` matches a string only if the letters `abc` occur as the first three characters of a line. The `%` symbol does not have its special metacharacter role if it is not at the beginning of a line.
`$`	Matches the end of a line. For example, xyz$ matches a string only if the letters `xyz` occur as the last three characters on the line. The `$` symbol does not have its special metacharacter role if it is not at the end of a line.
`@``c`	Escapes the following character (`c`). When followed by any metacharacter, the expression matches the metacharacter itself. For example, `@%` matches a percent sign (which otherwise would be interpreted as part of an expression specified as starting at the beginning of a line). Do not confuse this with the backslash used on the command line to escape characters that have special meaning to the command interpreter.
`*`	Indicates multiple occurrences of the preceding character or expression. A single character followed by an asterisk is a regular expression that matches zero or more occurrences of that one character. If the expression has multiple matches, it chooses the longest leftmost string that permits a match. For example, in a line starting `aaabaa`, the expression `a*` would match `aaa`.
[`string`]	Indicates a group of characters. A nonempty string of characters enclosed in square brackets matches any one character in the string. If the first character is a caret `(^`), the regular expression matches any character except a newline character and the remaining characters in the string. Use a hyphen to indicate a range of consecutive ASCII characters, such as `[0-9]`.

Here are examples of metacharacter usage with -x and -p:

[aeiou]: Matches any single character that is a vowel.
[^a-zA-Z0-9]: Matches any nonalphanumeric character.
-x [a-zA-Z][a-zA-Z]*$: Do not forward any messages containing lines that end in words. The specification is to match an entire word, that is one containing one or more alphabetic characters.
-p (?*): Forward any messages with lines containing parentheses.
-x %gobbledegook$: Ignores any message with a line that consists solely of gobbledegook.

Field Lengths

If the length of a string-valued field in the input file goes beyond its maximum, the value is truncated. For example, if an input message contains the user ID Administrator, it would be truncated to Administ.

The maximum lengths of the fields are the following.


Field	Maximum Length
Subsystem Name	8
User ID	8
Hostname	20
Function name	40
Transaction Key	21
Timestamp	20
Message Body	2000

Specifying the Date Format (%f)

When specifying the date by format, use the date format detailed in Table 4-4.

**Table 4-4 Date Formats**
Format	Explanation
`%%`	A literal percent sign
`%a`	Abbreviated weekday name (for example, `Sun`)
`%A`	Full weekday name (for example, `Sunday`)
`%b`	Abbreviated month name of the locale (for example, `Jan`)
`%B`	Full month name of the locale (for example, `January`)
`%d`	Day of month (1-31; leading zeroes are permitted but not required)
`%D`	Date as `%m/%d/%y`
`%h`	Same as `%b`
`%H`	Hour (0-23; leading zeroes are permitted but not required)
`%I`	Hour (0-12; leading zeroes are permitted but not required)
`%j`	Day number of year (001-366; leading zeroes are permitted but not required)
`%m`	Month number (1-12; leading zeroes are permitted but not required)
`%M`	Minute (0-59; leading zeroes are permitted but not required)
`%p`	Locale's equivalent of AM or PM
`%r`	Time as `%I:%M:%S %p`
`%S`	Seconds (0-59; leading zeroes are permitted but not required)
`%T`	Time as `%H:%M:%S`
`%y`	Year within century (0-99; leading zeroes are permitted but not required)
`%Y`	Year, including century (for example, `1998`)

Filtering a System Log

This section gives an example of filtering a system log. The following file contains messages from the UNIX system log (/var/log/syslog):

May 15 11:06:02 eclipse vmunix: psig: "EM_client" signal 15 was
masked, put back.

May 16 13:51:11 eclipse lpd[8951]: /usr/spool/lpd/lpd-log: No such
file or directory

May 17 10:38:12 eclipse su: 'su webuild' failed for emilie on
/dev/ttyp4

May 17 13:54:28 eclipse vmunix: NFS write error: on host iseult
remote file system full

May 17 13:54:37 eclipse last message repeated 13 times

May 17 14:40:42 eclipse lpd[9290]: /usr/spool/lpd/lpd-log: No such
file or directory

May 17 17:08:09 eclipse su: 'su root' succeeded for emilie on
/dev/ttyp0

Our configuration file (conf_file) contains the following lines:

-M LM_VERBOSE -D "%F1%V %F2%V %F3%f%h %d %T" -S " " -m NFS -d 123
-o %F4 -p "write error" -u emilie -n %F5 -b %F6-

-D "%F1%V %F2%V %F3%f%h %d %T" -S " " -m AUTH -d 124 -o %F4 -p su:
-u emilie -n %F5 -b %F6-

-D "%F1%V %F2%V %F3%f%h %d %T" -S " " -m PRINT -d 125 -o %F4 -p lpd
-u emilie -n %F5 -b %F6-

To filter the UNIX system log, run the following command:

log_monitor -i /var/log/syslog -f /home/emilie/conf_file

(For log_monitor command options, refer to Appendix 8, "Starting and Stopping Log Central.")

This produces the following Log Central messages:

|N|May 16 13:51:11
1998|PRINT|125|eclipse|11593|emilie|lpd[8951]:|0|1!/usr/spool/lpd
/lpd-log: No such file or directory

|N|May 17 10:38:12 1998|AUTH|124|eclipse|11593|emilie|su:|0|1!'su 
webuild' failed for emilie on /dev/ttyp4

|V|May 17 13:54:28 
1998|NFS|123|eclipse|11593|emilie|vmunix:|0|1!NFS write error: on 
host iseult remote file system full

|N|May 17 14:40:42 
1998|PRINT|125|eclipse|11593|emilie|lpd[9290]:|0|1!/usr/spool/lpd
/lpd-log: No such file or directory

|N|May 17 17:08:09 1998|AUTH|124|eclipse|11593|emilie|su:|0|1!'su 
root' succeeded for emilie on /dev/ttyp0

The following table shows how the mapping in the second line in conf_file produced the second Log Central message shown in the preceding. Two generated lines of output (the second and fifth messages in the preceding) were produced as a result of that mapping.


Input	Format Specification	Meaning	Output
`su`:	`-p su:`	Each message containing the specified string (`su:` in this case) is mapped according to the specifications in the second line of `conf_file`.	Log Central sample messages 2 and 5 (shown preceding this table).
None.	None provided.	Reporting mode; uses the default, `LM_NORMAL`, which produces an `N` in the Log Central file.	`N`
`May 17 14:40:42`	`-D "%F1%V %F2%V %F3%f%h %d %T" -S " "`	Time stamp; constructed from fields 1, 2, and 3.	`May 17 14:40:42 1998`
Field not present in input.	`-m AUTH`	Subsystem name; specified by literal value.	`AUTH`
Field not present in input.	`-d 124`	Message ID; specified by literal value.	`124`
`eclipse`	`-o %F4`	Host name; taken from field 4.	`eclipse`
Field not present in input.	None.	Process ID; uses default (PID of the Log Monitor process), since there is no `-I` specification.	`11593`
Field not present in input.	`-u emilie`	User name; specified by literal value.	`emilie`
`su:`	`-n %F5`	Function name; extracted from field number 5.	`su:`
Field not present in input.	None.	Transaction ID; uses default (0).	`0`
Field not present in input.	None.	Reserved.	`1!`
`'su webuild' failed for emilie on /dev/ttyp4`	`-b %F6-`	Message body; constructed from field 6 until the end of line.	`'su webuild' failed for emilie on /dev/ttyp4`

Note: There must be separators between the date format specifiers with %f; the same separator should appear in the value specified for -D. This is why %V is used in the first part of the specification to fill in the corresponding separators between the fields generated by Log Monitor.

Where fields are specified, the field numbers are calculated by using the separator specified with the -S option, which is a blank in the preceding example (the second line in conf_file).

Specifying Date Format

To specify the date format, precede the data format with %f.

If the input file contains lines like the following:

May 15 11:06:02 eclipse vmunix: psig: "EM_client" signal 15 was masked, put back.

May 16 13:51:11 eclipse lpd[8951]: /usr/spool/lpd/lpd-log: No such file or directory

To map these dates, you could use the following:

-D "%F1%V %F2%V %F3%f%h %d %T" -S " "

If the input file contains a line like the following:

eclipse|su:|12/12/99 09:20|'su root' succeeded for emilie on /dev/ttyp0

To map the date, you could use the following:

-D %F3%f"%D %H:%M" -S|

Converting Input Dates

The following rules are applied for converting the input specification into the internal format:

If only the weekday is given (that is, without specifying a day of the month), today is assumed if the given day is equal to the current day. Otherwise, the corresponding day from the next week is assumed.
If only the month is given (that is, without specifying a year), the current month is assumed if the given month is equal to the current month. Otherwise, the corresponding month from the next year is assumed. The first day of the month is assumed if no day is given.
If only the time is given (that is, without specifying a date), today is assumed if the given hour is greater than the current hour. Otherwise, the corresponding time from tomorrow is assumed. If no hour, minute, and second are given, the current hour, minute, and second are assumed.

The following examples illustrate these rules. Assume that the current date is Tue Sep 22 12:19:47 PDT 1998.

Input              Line in Template               Date
 ______________________________________________________________
 Tue                %a                    Sep 22 12:19:47 PDT 1998
 Mon                %a                    Sep 28 12:19:47 PDT 1998
 Fri                %a                    Sep 26 12:19:47 PDT 1998
 September          %B                    Sep 1 12:19:47 PDT 1998
 January            %B                    Jan 1 12:19:47 PST 1999
 December           %B                    Dec 1 12:19:47 PST 1998
 Sep Tue            %b %a                 Sep 1 12:19:47 PDT 1998
 Jan Sat            %b %a                 Jan 2 12:19:47 PST 1999
 Dec Tue            %b %a                 Dec 1 12:19:47 PST 1998
 Jan Fri 2003       %b %a %Y              Jan 3 12:19:47 PST 2003
 Fri 9              %a %H                 Sep 26 09:00:00 PDT 1998
 Feb 10:30          %b %H:%S              Feb 1 10:00:30 PST 1999
 10:30              %H:%M                 Sep 23 10:30:00 PDT 1998
 13:30              %H:%M                 Sep 22 13:30:00 PDT 1998