Reading LogRecords From A Log File or Table

The log writing sample program LogSample.java is fairly straightforward in the way the program writes a single record to a file or table as determined by the Logging Service's configuration. In contrast, the log reading sample program is more complex because you can specify that queries are applied to multiple files or tables.

Log files and tables in particular can become very large. If you specify multiple logs in a single query, create queries that are very specific, or limited in the number of records to return, or both specific and limited. If a large number of records are returned, the Access Manager resource limits (including those of the hosting system) may be exceeded.

LogReaderSample.java requires three command-line arguments which are used to authenticate with the Access Manager server. If you specify a log name, then the sample becomes a single-log reading application. If you don't specify a log name, reading from multiple logs is allowed. Reading from multiple logs does not preclude reading from a single log. Reading from multiple logs is useful when the exact log names available are unknown. The log reading sample is also very interactive. The following command-line example uses the LogReaderSample script:

./RunLogReader -o dc=iplanet,dc=com -u amadmin -p mypassword

In LogReaderSample.java, the command-line arguments are read. The following arguments are used to obtain the SSOToken that is specified in invoking the various LogReader.read() methods:

-o

organization name

-u

userID

-p

userID password

The LDAP login utility ldapLogin() is provided in a separate file, LogSampleUtils.java.

Next, the Logging Service configuration is read to determine, for example, whether file or database logging is specified and which log fields are logged.

manager.readConfiguration();     
String logStorageType = manager.getProperty(LogConstants.BACKEND);

Depending on whether Access Manager Logging Service is logging to a file or to a database, when the LogReader.getSize() method is invoked on a particular log name, LogReader.getSizeUnits() will return either LogConstants.NUM_BYTES or LogConstants.NUM_RECORDS. For example:

 i3 = LogReader.getSizeUnits(); 

The LogConstants.LOG_FIELDS property specifies which log fields have been specified for inclusion in the log record. For example:

String selFldsStr = manager.getProperty(LogConstants.LOG_FIELDS);

The time and Data fields are mandatory, thus they do not appear in the Logging Service list. They must be explicitly added to the Set of Fields to Retrieve.

	StringTokenizer stoken = new StringTokenizer(selFldsStr, ", ");
					String [] sFields = new String[stoken.countTokens() + 3];     
					Set allFields = new HashSet();

					allFields.add("time");
					allFields.add("data");

To get the Set of Log Names Available to read and their sizes:

	Set filesThereAre = LogReader.getLogNames();
for (Iterator it=filesThereAre.iterator(); it.hasNext(); ) {
	String fileName = (String)it.next();
	long li = 0;
	try {
		li = LogReader.getSize(fileName);
	} catch (Exception ex) {             
		System.out.println("got exception on file " +
				fileName + ". " + ex.getMessage());
	}         
	System.out.println (fileOrTable + " " + (i2++) +
		" = " + fileName + " contains " + li + " " +
		sizeUnit + ".");     
}

LogReaderSample.java allows you to select reads on a single or multiple logs. If a log name was specified on the command line with the -n option, then you can select from among the following types of reads:

	
			1. read all records
			2. specify logType
			3. specify logType and timeStamp
			4. specify logType and logQuery
			5. specify logType, timeStamp, and logQuery
			6. specify logQuery

If no log name was specified on the command line, and you select single log to read, you may select from only a list of pre—configured reports:

	Single (s) or multiple (m) file/table read: [s]
			What type of audit report to generate:
			 1. all records from file/table
			 2. authentication successes
			 3. authentication failures
			 4. login/logout activity
			 5. policy allows
			 6. policy denies
			 7. amAdmin CLI activity
			 8. amAdmin console activity
			 9. Federation access
			10. Federation errors
			11. Liberty access
			12. Liberty errors
			13. SAML access
			14. SAML error          
					enter type [1..14]: 

If you want to read from a selected single log, but specify the logQuery settings, do not use the -n command-line option. Select multiple log read, and then select the single log from which to read:

	Available files:
			file 0 = amAuthentication.access contains 1595 bytes.
			file 1 = amPolicy.access contains 2515 bytes.
			...
			file 13 = amAuthentication.error contains 795 bytes.

			Single (s) or multiple (m) file/table read: [s] m

			Available files:
			0: amAuthentication.access			
			1: amPolicy.access
			...
			12: amConsole.access-1					
			13: amAuthentication.error

			Enter selections (space-separated): 0
			What type of read to use: 
				1. read all records
				2. specify logQuery          
					 enter type [1 or 2]:  

The following table provides brief descriptions of the LogReader.read() methods.

read(String fileName,
         Object userCrdential)

read(String logName,
         String logType,
         Object userCrdential)

read(String logName,
	String logType,
	String timeStamp,
	Object userCrdential)

read(String logName,
	String logType,
	LogQuery logQuery,
	Object userCrdential)

read(String logName,
	String logType,
	String timeStamp,
	LogQuery logQuery,
	Object userCrdential)

read(String logName,
	LogQuery logQuery,
	Object userCrdential)

read(String logName,
	Set fileNames,
	LogQuery logQuery,
	Object userCrdential)

The LogQuery, along with the QueryElements that may be specified, are constructed in the getLogQuery() routine in LogReaderSample.java.

The following are brief descriptions of the LogQuery constructors.

LogQuery()

Creates a new LogQuery object with the following default values:

maxRecord = 
	LogQuery.MOST_RECENT_MAX_RECORDS 	 
	globalOperand = 
	LogQuery.MATCH_ANY_CONDITION 	    
queries = null (QueryElement) 	    
columns = null (columns to return)
sortBy = null (field to sort on)

LogQuery(int max_record)

Creates a new LogQuery object with the following values:

maxRecord = max_record
globalOperand = 	LogQuery.MATCH_ANY_CONDITION
queries = null (QueryElement)
columns = null (columns to return)
sortBy = null (field to sort on)

LogQuery(int max_Record, int matchCriteria, java.lang.String sortingBy)

Creates a new LogQuery object with the following values:

maxRecord = max_Record 	    
globalOperand = matchCriteria 	    
queries = null (QueryElement) 	    
columns = null (columns to return) 	    
sortBy = sortingBy (field to sort on)

The LogQuery object created with the constructors may be subsequently modified with the following set* methods:

• setColumns(java.util.ArrayList columns)

• setGlobalOperand(int no)

• setMaxRecord(int value)

• setSortingField(java.lang.String fieldName)

A LogQuery may specify a List of QueryElements, each containing a value for a field (column) and a relationship. The following sample code queries for all successful authentications in domain dc=iplanet,dc=com, and returns the time, Data, MessageID, ContextID, LoginID, and Domain fields, sorted on the LoginID field:

ArrayList al = new ArrayList();
al.add (LogConstants.TIME);
al.add (LogConstants.Data);
al.add (LogConstants.MESSAGE_ID);
al.add (LogConstants.CONTEXT_ID);
al.add (LogConstants.LOGIN_ID);
al.add (LogConstants.DOMAIN);
LogQuery lq = new LogQuery(LogQuery.ALL_RECORDS,
		LogQuery.MATCH_ALL_CONDITIONS,
		LogConstants.LOGIN_ID);

QueryElement qe1 = new QueryElement(LogConstants.MESSAGE_ID,
		"AUTHENTICATION-105",
		QueryElement.EQ);
lq.addQuery(qe1);

QueryElement qe2 = new QueryElement(LogConstants.DOMAIN,
		"dc=iplanet,dc=com",
		QueryElement.EQ);
lq.addQuery(qe2);

QueryElement supports the following relationships:

QueryElement.GT

Greater than

QueryElement.LT

Less than

QueryElement.EQ

Equal to

QueryElement.NE

Not equal to

QueryElement.GE

Greater than or equal to

QueryElement.LE

Less than or equal to

QueryElement.CN

Contains

QueryElement.SW

Starts with

QueryElement.EW

Ends with

In the example, assuming that dc=iplanet,dc=com is the root domain, changing the qe2relationship field to QueryElement.EW (Ends with) or QueryElement.CN (Contains) changes the query to include all successful authentications in all domains. To read the example query from the amAuthentication.access log, assuming the SSOToken is in ssoToken:

 String[][] result = new String[1][1];
    result = read("amAuthentication.access", lq, ssoToken);

The first record (row 0) contains the field and column names. See the printResults() method in LogReaderSample.java for a sample display routine.