XML File Collection Plug-ins

Learn how to use Oracle DBSecCentral XML file collection plug-ins to collect audit data from an XML file type of trail.

XML file collection plug-ins support collection of audit data from an XML file type of trail. All these XML audit files must be present in single directory. You can specify details of the XML audit data in the mapper file. This XML mapper file must conform to the schema.

Related Topics

• Schema For XML File Collection Plug-in Mapper File

Requirements for XML File Collection Plug-ins

To use XML collection plug-ins for reading audit trails from XML files, your data must meet Oracle Database Security Central requirements.

You can use collection plug-ins for reading audit trails from XML audit record files if the XML files meet the requirements for collection.

XML File Audit Record File Requirements for Oracle Database Security Central

• The audit trail must be stored in one or more XML files in a single directory path.

• The user must have read permission on the directory containing the XML audit files.

• XML files in this directory must be valid, well-formed XML documents, within the constraints of the XML 1.0 specification.

• The file and record start elements must be as specified in the mapper file.

• All the audit record elements should be at the same level in Audit XML files.

• All the audit record elements in Audit XML files must be the same.

• Under every audit record element, all the field elements must be at the same level and one level below the audit record start element.

• The XML audit file must have an element value that can be mapped to the CommandClass core field. If a record has its CommandClass field as null, then the record is treated as invalid.

• The XML audit file must have an element value that can be mapped to the UserName core field. If a record has its UserName field as null, then the record is treated as invalid.

• In the XML file, each audit record must have a timestamp as one of its element values.

  The value of the timestamp element must be monotonically increasing, that is, the value of the field increases with every new audit record inserted into the trail. The timestamp value should be strictly Not Null. Timestamp format must be according to SimpleDateFormat Java class.

  This field must mapped to the EventTimeUTC core field in the mapper file. If mapping for event time is not specified in the mapper file, then the collection plug-in shuts down. If the field value for the event time in audit records is found null, then the collection plug-in takes the time of the record last sent from the same XML audit file.

• The audit trail must contain a single element value or group of element values in the audit record that uniquely identify each audit record in XML Audit files.

• Common information shared by all audit records in XML file should be present in the beginning of the XML file, under the file start element, at the same level as the audit record elements.

• If an audit data target produces audit files with multiple XML formats, then the user must provide a separate mapper file for each audit file format having a different start element.

• XML files in this directory should be of the same locale and encoding as the agent, as described in the examples below:

  • Valid: The user has an agent in a Chinese locale (env). XML files are also generated in a Chinese locale with same encoding (for example, ZHS16GBK). This setup is valid.

  • Invalid: The user has an agent in a German locale (env). XML files are generated/moved from some other computer, which are Chinese encoded. The collectors fail to start because of an encoding mismatch, as well as a locale mismatch, in this case. This setup is invalid.

Example Audit Trail for an XML File Collection Plug-in

This example audit trail for an xml file collection plug-in shows the details of an XML file collection plug-in.

This example file is used in other locations to demonstrate the creation and structure of a sample mapper file for the creation and structure of a sample mapper file for an XML file collection plug-in in Oracle Database Security Central documentation.

The following table lists the audit record structure and mappings to Oracle Audit Vault Server fields for the hypothetical target type, XMLSOURCE, which generates and stores audit data in XML audit files.

Table 2 Audit Data Fields in XML Audit Records and Mappings

Target Field	Audit Vault Server Field	Map to Field Type
USER_ID	UserName	core field
OS_USER_ID	OSUserName	core field
ACTION	CommandClass	core field
STATUS	EventStatus	core field
EVENT_TIME	EventTimeUTC	core field
OBJ_NAME	TargetObject	core field
OBJ_CREATOR	TargetOwner	core field
USER_HOST	ClientHostName	core field
SQL_TEXT	CommandText	core field
SQL_BIND	CommandParam	core field
TERMINAL	TerminalName	extension field
DB_ID	extension field	extension field
INSTANCE	extension field	extension field
PROCESS	extension field	extension field
SESSION_ID	marker field	marker field
ENTRY_ID	marker field	marker field

Example 1 Sample XML Audit Record

<?xml version="1.0" encoding="UTF-8"?>
<Audit>
    <AuditRecord>
        <Audit_type>1</Audit_type>
        <User_id>scott</User_id>
        <Os_user_id>usr1</Os_user_id>
        <Action>select</Action>
        <Status>0</Status>
        <Event_time>2010-11-11 12:23:59.166</Event_time>
        <Obj_name>emp</Obj_name>
        <Terminal>t1</Terminal>
        <Db_id>136</Db_id>
        <Session_id>170191</Session_id>
        <Entry_id>1</Entry_id>
    </AuditRecord>
    <AuditRecord>
        <Audit_type>3</Audit_type>
        <User_id>scott</User_id>
        <Os_user_id>usr1</Os_user_id>
        <Action>delete</Action>
        <Status>1</Status>
        <Event_time>2010-11-11 12:33:59.166</Event_time>
        <Obj_name>emp</Obj_name>
        <Terminal>t1</Terminal>
        <Db_id>136</Db_id>
        <Session_id>170191</Session_id>
        <Entry_id>2</Entry_id>
    </AuditRecord>
</Audit>

Creating the XML File Audit Collection Mapper File

To create an XML file collection plug-in mapper file, you must describe the collection plug-in mappings in this mapper file in accordance with Oracle Database Security Central standards.

You must describe the collection plug-in mappings in this mapper file as follows:

Standards for Collection Plug-in Mappings in Mapper Files for Oracle Database Security Central

• Top-Level Element

  <AVXMLCollectorTemplate securedTargetType="XMLSOURCE"
      maxSecuredTargetVersion="11.0" version="1.0">

  The AVXMLCollectorTemplate is the top level element and has these mandatory attributes: securedTargetType, maxSecuredTargetVersion, and version. The minSecuredTargetVersion attribute is optional.

  The accepted format for the minSecuredTargetVersion, maxSecuredTargetVersion, and version attributes uses numbers, separated by dots, such as 12.2,10.3.2, 11.2.3.0.

• Header Information

  <HeaderInfo>
    <StartTag>Audit</StartTag>
  </HeaderInfo>

  HeaderInfo is mandatory. It contains one child element, StartTag, which names the top-level element of the audit record file.

• Record Information

  <RecordInfo>
    <StartTag>AuditRecord</StartTag>
  </RecordInfo>

  RecordInfo provides the starting element of audit records in XML audit files. RecordInfo is mandatory.

  StartTag is the starting element of each audit record in XML audit files.

• Field Mapping Information

  <FieldMappingInfo>

  FieldMappingInfo provides mapping information from target fields to various Audit Vault fields, contained in these child elements, CoreFields, LargeFields, ExtensionField, and MarkerField.

  Field mappings include <Map> elements, which contain <Name> elements that hold target field names, and <MapTo> elements that hold Audit Value field names that targets are mapped to.

  There should be no many-to-one mappings from target fields to Audit Vault Server fields. For example, the following is invalid:

  <!-- Invalid code
  <Map>
      <Name>USER_ID</Name>
  
      **<MapTo>UserName</MapTo>**
  </Map>
  <Map>
  
      **<Name>OS_USER_ID</Name>**
  
      **<MapTo>UserName</MapTo>**
  </Map>  -->

  • Core Fields

    <CoreFields>

    CoreFields provides mapping from target fields to core fields of Audit Vault Server. Target fields specified in core field mappings must be of SQL data type, either a string or a data type that can convert to string.

    The following elements contain core fields.

    <Map>
     <Name>EVENT_TIME</Name>
      <MapTo>EventTimeUTC</MapTo>
      <TimestampPattern>yyyy-MM-dd HH:mm:ss.SSS</TimestampPattern>
    </Map>

    EventTimeUTC provides event time mapping information. The value in TimestampPattern specifies the timestamp format for event time. EventTimeUTC and TimestampPattern are mandatory.

    When specifying the TimestampPattern, use the supported patterns and characters of the Java SimpleDateFormat class, NOT Oracle Database specific patterns.

    For multibyte characters such as Chinese, specific words such as Month should be added into the pattern as characters in SimpleDateFormat. The AM and PM indicators are obtained based on locale, but should be explicitly mentioned in the TimestampPattern that you provide in the mapper file.

    <Map>
      <Name>USER_ID</Name>
      <MapTo>UserName</MapTo>
    </Map>

    UserName represents the user who performed the action. If the mapping is not provided, Audit Data Collection still starts successfully, but every audit record is treated as invalid.

    <Map>
      <Name>OS_USER_ID</Name>
      <MapTo>OSUserName</MapTo>
    </Map>
     <Map>
    
      **<Name>ACTION</Name>**
      <MapTo>CommandClass</MapTo>
    </Map>

    CommandClass represents the action of the event. If the mapping is not provided, Audit Data Collection still starts successfully, but all audit records are treated as invalid.

    <Transformation>
      <ValueTransformation from="1" to="CREATE"/>
      <ValueTransformation from="2" to="INSERT"/>
      <ValueTransformation from="3" to="SELECT"/>
      <ValueTransformation from="4" to="CREATE"/>
      <ValueTransformation from="15" to="READ"/>
      <ValueTransformation from="30" to="LOGON"/>
      <ValueTransformation from="34" to="LOGOFF"/>
      <ValueTransformation from="35" to="ACQUIRE"/>
    </Transformation>

    CommandClass contains a Transformation field with ValueTransformation values, from targets to the Audit Vault Server CommandClass field. These transformations are mandatory.

    The to attributes are values for the CommandClass field. If you can meaningfully map an event to one of these values, Oracle recommends that you do so. If this is not possible, use a value that appropriately reflects the action that generated the audit event.

    <Map>
      <Name>OBJ_NAME</Name>
      <MapTo>TargetObject</MapTo>
    </Map>
    <Map>
      <Name>USER_HOST</Name>
      <MapTo>ClientHostName</MapTo>
    </Map>
    
    <Map>
       <Name>TERMINAL</Name>
       <MapTo>TerminalName</MapTo>
    </Map>
    <Map>
      <Name>OBJ_CREATOR</Name>
      <MapTo>TargetOwner</MapTo>
    </Map>
    <Map>
      <Name>STATUS</Name>
      <MapTo>EventStatus</MapTo>
            <Transformation>
              <ValueTransformation from="0" to="FAILURE"/>
              <ValueTransformation from="1" to="SUCCESS"/>
              <ValueTransformation from="2" to="UNKNOWN"/>
            </Transformation>
    </Map>

    EventStatus contains a Transformation field with ValueTransformation values, from targets to Audit Vault EventStatus fields. These transformations are mandatory.

    </CoreFields>

  • Large Fields Information

    <LargeFields>
      <Map>
        <Name>SQL_TEXT</Name>
        <MapTo>CommandText</MapTo>
      </Map>
    <Map>
       <Name>COMMAND_PARAMETER</Name>
       <MapTo>CommandParam</MapTo>
      </Map>
    </LargeFields>

    LargeFields are target fields mapped to large fields in the Audit Vault Server. The specified target fields must be of SQL data type CLOB or String, or be convertible to String.

  • Extension Fields

    <ExtensionField>
       <Name>DB_ID</Name>
       <Name>INSTANCE</Name>
       <Name>PROCESS</Name>
    </ExtensionField>

    ExtensionFields are target field names that must be stored as a name-value pair in the Extension field in Audit Vault Server. Target fields specified must be of SQL data type CLOB or String, or be convertible to String.

  • Marker Fields

    <MarkerField>
      <Name>SESSION_ID</Name>
      <Name>ENTRY_ID</Name>
    </MarkerField>

    MarkerField contains a list of target fields that uniquely identify each audit record. The target fields specified must be of SQL data type CLOB or String, or be convertible to String. MarkerField is mandatory.

See Also:

• XML File Collection Plug-in Examples

• Core Fields

• Actions and Target Types

• Large Fields

• Extension Field

• Marker Field

XML Transformation for Non-Standard Audit Records

If you have audit records in a non-standard audit data format, you can apply XML transformation using XSL on the XML audit records.

To apply XML transformation on the audit records, you provide an XSL file that can transform the audit data from its original format to the format currently specified for the XML file collection plug-ins. Doing this means that you can enhance file collection plug-ins to support a variety of XML audit data formats.

Related Topics

• Example Audit Trail for an XML File Collection Plug-in

Additional Requirement for XML Transformation Using XSL

To transform non-standard audit records into the current format, your transformer must follow Oracle Database Security Central standards.

The transformer must write to audit files in an incremental order. That is, the transformer must write to one audit file until its maximum size is reached, and then move over to another file. Therefore, only one file can be active at a time. If the transformer finds more than one incomplete XML audit file, then the XML file collection plug-in stops.

Changes Required to Transform Non-Standard Audit Records

To transform non-standard audit records with Oracle Database Security Central, you must complete this procedure.

You must perform these steps:

1. Add a section such as this example to the mapper file after <RecordInfo>, specifying the name of XSL file that you want to be used for transformation, and the SourceFileStartTag for the file to be transformed.

   <XslTransformation>
       <XslFile>test_template.xsl</XslFile>
       <SourceFileStartTag>AUDIT</SourceFileStartTag>
   </XslTransformation>

2. Provide the XSL file and place it in the templates folder of the plugin directory.

3. You can also make calls to Java functions from within the XSL file. To do this, place the jar file created in the jars folder of the plugin directory.

Related Topics

• Creating the XML File Audit Collection Mapper File

• Sample Non-Standard XML Audit Data Record

• Creating an XSL File for Transformation

Sample Non-Standard XML Audit Data Record

See how to transform an XML data record to the proper XML format required for an XML file collection plug-in.

As you review this example, note that your source system can produce audit records with a different appearance.

Example 2 Audit.xml: Sample XML Audit Record

<?xml version="1.0" encoding="UTF-8"?>
<AUDIT>

   <AUDIT_RECORD TIMESTAMP="2013-06-07T08:27:53" NAME="Audit"
   SERVER_ID="0" VERSION="1" STARTUP_OPTIONS="C:/Program Files/MySQL/MySQL
   Server 5.6/bin\mysqld --defaults-file=C:\ProgramData\MySQL\MySQL Server
   5.6\my.ini" OS_VERSION="x86_64-Win64" MYSQL_VERSION=
   "5.6.11-enterprise-commercial-advanced"/>

   <AUDIT_RECORD TIMESTAMP="2013-06-07T08:30:46" NAME="Connect" CONNECTION_ID="1"
   STATUS="0" USER="root" PRIV_USER="root" OS_LOGIN="" PROXY_USER=""
   HOST="localhost" IP="127.0.0.1" DB=""/>

   <AUDIT_RECORD TIMESTAMP="2013-06-07T08:31:21" NAME="Query" CONNECTION_ID="1"
   STATUS="0" SQLTEXT="CREATE USER 'admin'@'localhost' IDENTIFIED BY
   'welcome_1'"/>

</AUDIT>

Creating an XSL File for Transformation

To create an XSL transformation file that defines transformation rules you must create a version that can transform the source audit records that your system creates, and place it in the templates folder of the plugin.

The Audit.xml transformed audit record file does not appear in your folder. It is just an example showing the result of transforming the Audit.xml file into the required XML format, using the XSL transformation file in the test_template.xsl example.

Example 3 test_template.xsl

<?xml version="1.0"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output indent="yes" />
     <xsl:template match="/">
        <ROOT_DEST>
          <xsl:for-each select="AUDIT/AUDIT_RECORD">
            <Record_Dest>
                <USER><xsl:value-of select="@USER"/></USER>
                <PRIV_USER><xsl:value-of select="@PRIV_USER"/></PRIV_USER>
                <OS_LOGIN><xsl:value-of select="@OS_LOGIN"/></OS_LOGIN>
                <PROXY_USER><xsl:value-of select="@PROXY_USER"/></PROXY_USER>
                <HOST><xsl:value-of select="@HOST"/></HOST>
                <IP><xsl:value-of select="@IP"/></IP>
                <DB><xsl:value-of select="@DB"/></DB>
                <SQLTEXT><xsl:value-of select="@SQLTEXT"/></SQLTEXT>
                <CONNECTION_ID><xsl:value-of select=
                      "@CONNECTION_ID"/></CONNECTION_ID>
                <STATUS><xsl:value-of select="@STATUS"/></STATUS>
                <TIMESTAMP><xsl:value-of select="@TIMESTAMP"/></TIMESTAMP>
                <NAME><xsl:value-of select="@NAME"/></NAME>
                <SERVER_ID><xsl:value-of select="@SERVER_ID"/></SERVER_ID>
                <VERSION><xsl:value-of select="@VERSION" /></VERSION>
   <STARTUP_OPTIONS><xsl:value-of select="@STARTUP_OPTIONS"/> </STARTUP_OPTIONS>
                <OS_VERSION><xsl:value-of select="@OS_VERSION"/></OS_VERSION>
                <MYSQL_VERSION><xsl:value-of select="@MYSQL_VERSION"/>
                    </MYSQL_VERSION>
            </Record_Dest>
          </xsl:for-each>
         </ROOT_DEST>
        </xsl:template>
  </xsl:stylesheet>

Example 4 Transformed Audit Record file

<ROOT_DEST>
     <Record_Dest>
         <USER></USER>
         <PRIV_USER></PRIV_USER>
         <OS_LOGIN></OS_LOGIN>
         <PROXY_USER></PROXY_USER>
         <HOST></HOST>
         <IP></IP>
         <DB></DB>
         <SQLTEXT></SQLTEXT>
         <CONNECTION_ID></CONNECTION_ID>
         <STATUS></STATUS>
         <TIMESTAMP>2013-06-07T08:27:53</TIMESTAMP>
         <NAME>Audit</NAME>
         <SERVER_ID>0</SERVER_ID>
         <VERSION>1</VERSION>
         <STARTUP_OPTIONS>C:/Program Files/MySQL/MySQL Server 5.6/bin\mysqld
             --defaults-file=C:\ProgramData\MySQL\MySQL Server
             5.6\my.ini</STARTUP_OPTIONS>
         <OS_VERSION>x86_64-Win64</OS_VERSION>
         <MYSQL_VERSION>5.6.11-enterprise-commercial-advanced</MYSQL_VERSION>
     </Record_Dest>
     <Record_Dest>
         <USER>root</USER>
         <PRIV_USER>root</PRIV_USER>
         <OS_LOGIN></OS_LOGIN>
         <PROXY_USER></PROXY_USER>
         <HOST>localhost</HOST>
         <IP>127.0.0.1</IP>
         <DB></DB>
         <SQLTEXT></SQLTEXT>
         <CONNECTION_ID>1</CONNECTION_ID>
         <STATUS>0</STATUS>
         <TIMESTAMP>2013-06-07T08:30:46</TIMESTAMP>
         <NAME>Connect</NAME>
         <SERVER_ID></SERVER_ID>
         <VERSION></VERSION>
         <STARTUP_OPTIONS></STARTUP_OPTIONS>
         <OS_VERSION></OS_VERSION>
         <MYSQL_VERSION></MYSQL_VERSION>
     </Record_Dest>
     <Record_Dest>
         <USER></USER>
         <PRIV_USER></PRIV_USER>
         <OS_LOGIN></OS_LOGIN>
         <PROXY_USER></PROXY_USER>
         <HOST></HOST>
         <IP></IP>
         <DB></DB>
         <SQLTEXT>CREATE USER 'admin'@'localhost' IDENTIFIED BY
                 'welcome_1'</SQLTEXT>
         <CONNECTION_ID>1</CONNECTION_ID>
         <STATUS>0</STATUS>
         <TIMESTAMP>2013-06-07T08:31:21</TIMESTAMP>
         <NAME>Query</NAME>
         <SERVER_ID></SERVER_ID>
         <VERSION></VERSION>
         <STARTUP_OPTIONS></STARTUP_OPTIONS>
         <OS_VERSION></OS_VERSION>
         <MYSQL_VERSION></MYSQL_VERSION>
     </Record_Dest>
</ROOT_DEST>