2 Drop Folder Monitor Solutions and Basic Configurations

This chapter describes details and configurations of the different DFM solutions, and includes the following information:

DFM Solution for Single Mode Folders

This DFM Solution can monitor a local disk, FTP transfers, and CIFS folders. DIVArchive will archive each file found in the Drop Folder as a single object. As files are dropped into the folder they are archived as single files into DIVArchive. Multiple files can be in the Drop Folder at the same time, but they will be handled by the DFM one at a time.

Single File Mode Workflow

Supported Source/Destinations include:

  • Local disks

  • FTP folders

    When the Drop Folder is configured on an FTP server, you must copy all content (files and folders to be archived) to the Drop Folder using FTP. If you copy the content to the Drop Folder using another process (local copy through the operating system, remote desktop copy, and so on), there is no guarantee DFM will process the content properly.

  • CIFS folders

    Linux-based Actors do not support UNC paths for CIFS sources and destinations.

The following are additional configuration parameters and functionality:

  • Only single file objects are supported.

  • Supported major configuration parameters include:

    • Oracle DIVArchive Manager

    • Drop Folder URL

    • Category Name of objects being archived

    • Source/Destination for the Archive request

    • Templates for File Path Root and File Name generation

    • DIVArchive Media Name for the Archive request

Configuring the Basic DFM Single Mode Folder Solution

In the basic DFM solution, DIVArchive will archive each file found in the Drop Folder as single object. You use the following procedure to configure this DFM solution for a Single Mode Folder. Editing the DFM configuration file is necessary to make the following changes.

  1. Rename the dfm.conf.single.local.ini file to dfm.conf.

    The file is located in the %DIVA_HOME%\Program\conf\dfm\ folder.

  2. Specify the Host and Port using <managerConnetion> as follows (using your parameters):

    <managerConnetion>
 <address host="localhost" port="9000"/>
</managerConnetion>

  3. Specify the Windows Service Name using <serviceName> as follows:

    <serviceName>DIVArchiveDFM</serviceName>

    You must write this parameter on one line, and there must not be any symbols or empty spaces at the beginning of the line.

  4. Specify the Folder URL using <url> as follows (using your URL):

    <url>file:///c:\DROPFOLDER\</url>

  5. Specify the Category Name of objects to be archived using <categoryName> as follows (using your Category Name):

    <categoryName>Category</categoryName>

  6. Specify the Source/Destinations to use in Archive requests using <sourceDestinationDIVAName> as follows using your Source/Destination name):

    <sourceDestinationDIVAName>DISK</sourceDestinationDIVAName>

  7. Specify the File Path to the folder containing the files DIVArchive will archive using <archiveFilePathTemplate> as follows (using your File Path):

    <archiveFilePathTemplate platform="DETECT" options="">
 URL_TO_FILE
</archiveFilePathTemplate>

  8. Specify the template for the generation of the File Name using <archiveFileNameTemplate> as follows (using your template name):

    <archiveFileNameTemplate platform="DETECT" options="">
 Filename.Ext
</archiveFileNameTemplate>

  9. Specify the DIVArchive Media Name used in the archive request using <divaMediaName> as follows (using your Media Name):

    <divaMediaName>Array1</divaMediaName>

    Caution:

    If you do not complete the following Step 10 the installation will fail because it cannot find the trace file.

  10. Rename the %DIVA_HOME%\Program\conf\dfm\dfm.trace.ini file to dfm.trace.

  11. Install the DFM service using the following command:

    %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat install

  12. Start the DFM service using the following command, or open the Windows Services panel and start the service from there.

    %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat start

DFM Solution for File Set Folders on Local Disk and FTP Servers

This DFM solution offers more flexibility than the basic solution. This configuration can monitor the local disk, FTP server, and CIFS folders that contain either single files or file sets.

DFM supports single and set based folder types. Each file dropped into a Single type folder results in a separate archive per file. File Set type folders provide a means of archiving objects comprised of multiple files. File Set mode archive operations require the use of metadata files, which contain the details of the file set comprising a specific archive operation (see Metadata File). Files for separate archive operations can coexist in a File Set mode folder, because the scope of files included in each archive operation is determined by the file list in each MDF file placed in the File Set mode folder.

Caution:

The last file created in the File Set Drop Folder must be the metadata file (with the .mdf extension). The presence of this file triggers the archive operation. If the metadata file is transferred before the actual data files, content loss can occur!

AXF format files are, at the file system level, single files, but contain many files internally. You can archive AXF files into DIVArchive, but this requires the use of a File Set mode folder and an MDF file. The MDF file contains the list of files within the AXF file, and the target parent attributes of the object used when archived.

See Appendix B for sample metadata files in DIVArchive DFM format.

File Set Mode Workflows

Supported Source/Destinations include:

  • Local disks

  • FTP folders

    When the Drop Folder is configured on an FTP server, you must copy all content (files and folders to be archived) to the Drop Folder using FTP. If you copy the content to the Drop Folder using another process (local copy through the operating system, remote desktop copy, and so on), there is no guarantee DFM will process the content properly.

  • Unsecured CIFS folders

    Secured CIFS is possible if the Windows DFM Service is run under a suitable user account.

The following are additional configuration parameters and functionality:

  • Single and multiple file objects are supported.

  • Recursive Archive (folders and files) is supported

  • Supported major configuration parameters include:

    • Oracle DIVArchive Manager

    • Drop Folder URL

    • DIVArchive Media Name for the Archive request

Configuring Recursive Archive

DFM Recursive Archive is enabled when the DFM folder is configured to use File Set Mode. In this mode the Archive File Set is defined in the specific object configuration .mdf file.

This feature defines a special character, asterisk, that can be used in the DFM metadata file (the .mdf file). Applying an asterisk to the end of the folder path causes all files under this path to be archived, but does not include subfolders.

Using the -r option in the configuration of the drop folder achieves different results than using the asterisk option. When you include the -r option in the request, the Oracle DIVArchive Actor will recurse into all subfolders whether the asterisk is used or not.

Conversely, if the -r option is not used, the Actor only selects the files in the immediate folder. In this case, use of the asterisk will still cause archiving of all files in the immediate folder action to occur. When using this option, be aware of the request and Source/Destination configurations to achieve the desired results.

You can configure the -r option in either of the two following locations:

  • In the request Options field when creating the request. This method is only applicable to the current request. You enter the -r option in the Options field to use this method.

  • In the Source/Destination Connect Options field in the Configuration Utility. This method is applicable to all requests to this Source/Destination. You include the -r option in the Connect Options field with your other request options to use this method.

Note:

Recursive Archive does consider the use of the -file_order FILE_FIRST and -file_order DIRS_FIRST command options.

The following are several simple examples (excerpts from the .mdf file), which enables recursive archive for the specific folders and files:

Example 1:

<fileList>
 folder1/*
 test/folder2/*
 folder3/test.txt
</fileList>

The results of this configuration are as follows:

  • Every file under folder1 and test/folder2 will be archived, but not subfolders.

    If you used the -r option in the request Options, or on the Source/Destination Configuration, all subfolders and their files are also archived.

  • For folder3/test.txt, only the test.txt file located in folder3 will be included.

Example 2:

<fileList>
 *
</fileList>

The results of this configuration are as follows:

  • Every file in the specified folder will be archived, but not subfolders.

  • If you used the -r option in the request Options, or on the Source/Destination Configuration, all subfolders and their files are also archived.

Example 3:

<fileList>
 folder1/*
</fileList>

The results of this configuration are as follows:

  • Every file under folder1 will be archived, but not subfolders.

  • If the -r option was used in the request Options, or on the Source/Destination Configuration, all folder1 subfolders and their files will also be archived.

Configuring the DFM Solution for File Set Folders

You use the following procedure to configure the DFM solution for a File Set folder, default filter for local disks, and FTP:

  1. Rename the dfm.conf.set.local.ftp.ini file to dfm.conf.

    The file is located in the %DIVA_HOME%\Program\conf\dfm\ folder.

  2. Specify the Host and Port using <managerConnetion> as follows (using your parameters):

    <managerConnetion>
 <address host="localhost" port="9000"/>
</managerConnetion>

  3. Specify the Windows Service Name using <serviceName> as follows:

    <serviceName>DIVArchiveDFM</serviceName>

    You must write this parameter on one line, and there must not be any symbols or empty spaces at the beginning of the line.

  4. Specify the <folderConfig> node using <type>. The folder should be the File Set type, and therefore the set value should be specified as follows:

    <type>set</type>

  5. Specify the Folder URL using <url> as follows (using your URL):

    <url>ftp://diva:diva@localhost:21/dropfolder</url>

  6. Specify the DIVArchive Media Name to use in the archive request using <divaMediaName> as follows (using your Media Name):

    <divaMediaName>Array1</divaMediaName>

    Caution:

    If you do not complete the following Step 7 the installation will fail because it cannot find the trace file.

  7. Rename the %DIVA_HOME%\Program\conf\dfm\dfm.trace.ini file to dfm.trace.

  8. Install the DFM service using the following command:

    %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat install

  9. Start the DFM service using the following command, or open the Windows Services panel and start the service from there.

    %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat start

DFM Solution for Delete Mode

This DFM solution monitors the specified folder, and removes obsolete files. If a subfolder is empty for more than the configured allowed time period, it will also be removed. This solution is used in with the other DFM solutions, and should be included in all DFM configurations.

DFM Delete Mode Workflow

The following major configuration parameters are supported:

  • Folder URL

  • Interval specifying when files will be deleted from the folder

Configuring the DFM Solution for Delete Mode

You use the following procedure to configure the DFM solution for Delete Mode:

  1. Rename the dfm.conf.delete.ini file to dfm.conf.

    The file is located in the %DIVA_HOME%\Program\conf\dfm\ folder.

  2. Specify the interval that identifies when the file will be deleted from the folder using <fileDeleteThreshold> as follows (the default value is shown):

    <fileDeleteThreshold>86400</fileDeleteThreshold>

    • This value is the File Delete Threshold in seconds, and for folders only in Delete Mode.

    • This interval specifies when a file will be deleted from a DFM folder after copying the file to the folder is complete.

    • The state of completion is identified by the file size. As the file is copied, the size of the file will continue to grow until the copy is complete. Once the file size no longer increases, the system recognizes that the file copy operation is complete.

    • This value must be a positive integer value.

    • The default value of 86400 seconds equals 24 hours.

  3. Specify the Windows Service Name using <serviceName> as follows:

    <serviceName>DIVArchiveDFM</serviceName>

    You must write this parameter on one line, and there must not be any symbols or empty spaces at the beginning of the line.

  4. Specify the <folderConfig> node using <mode> as follows:

    <mode>Delete</mode>

  5. Specify the Folder URL using <url> as follows (using your URL):

    <url>file:///c:\CLEAN_FOLDER\</url>

    Caution:

    If you do not complete the following Step 6 the installation will fail because it cannot find the trace file.

  6. Rename the %DIVA_HOME%\Program\conf\dfm\dfm.trace.ini file to dfm.trace.

  7. Install the DFM service using the following command:

    %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat install

  8. Start the DFM service using the following command, or open the Windows Services panel and start the service from there.

    %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat start

DFM Solution for Oracle DIVAnet

This DFM Solution monitors a local disk, and can use either Single Files or File Sets. Each metadata file found in the Drop Folder is parsed, and a new object (usually with more than one file) is archived in DIVArchive. DIVAnet systems configured with inter-site transfer capabilities use this solution.

See Appendix A for Oracle DIVArchive options and licensing information, and contact Oracle Support for detailed information about various available DIVAnet solutions.

DFM DIVAnet Solution Workflow

Supported Source/Destinations include:

  • Local disks

The following are additional configuration parameters and functionality:

  • Single and multiple file objects are supported.

    Files must be in non-complex objects.

  • Supported major configuration parameters include:

    • Oracle DIVArchive Manager

    • Drop Folder URL

    • DIVArchive Media Name Pattern for generation of the Media Name for the Archive request

    • Source/Destination name

    • Source/Destination path

Configuring the DFM Solution for Oracle DIVAnet

This DFM Solution monitors the local disk. Each metadata file found in the Drop Folder is parsed, and a new object (usually with more than one file but not a complex object) is archived in DIVArchive.

You use the following procedure to configure the DFM Solution for DIVAnet:

  1. Rename the dfm.conf.divanet.ini file to dfm.conf.

    The file is located in the %DIVA_HOME%\Program\conf\dfm\ folder.

  2. Specify the Host and Port using <managerConnetion> as follows (using your parameters):

    <managerConnetion>
 <address host="localhost" port="9000"/>
</managerConnetion>

  3. Specify the Windows Service Name using <serviceName> as follows:

    <serviceName>DIVArchiveDFM</serviceName>

    You must write this parameter on one line, and there must not be any symbols or empty spaces at the beginning of the line.

  4. Specify the <folderConfig> node using <type>. The folder should be the File Set type, and therefore the set value should be specified as follows:

    <type>set</type>

    There are two types of folders: Single and File Set. If the folder type should be Single, then you must specify the single value. If the folder type should be File Set, then you must specify the set value as shown in the example.

  5. Specify the Folder URL using <url> as follows (using your URL):

    <url>file:///c:\DROPFOLDER\</url>

  6. Specify the DIVArchive Media Name Pattern to use in the archive request using <divaMediaNamePattern> as follows (using your Media Name Pattern):

    <divaMediaNamePattern>/data/$GROUP/</divaMediaNamePattern>

  7. Specify the DIVArchive Source/Destination Name using <sourceDestinationDIVAName> as follows (using your Source/Destination name):

    <sourceDestinationDIVAName>SourceDestinationServer</sourceDestinationDIVAName>

  8. Specify the DIVArchive Source/Destination Path using <sourceDestinationDIVAPath> as follows (using your Source/Destination path):

    <sourceDestinationDIVAPath>SourceDestinationServer</sourceDestinationDIVAPath>

    Caution:

    If you do not complete the following Step 9 the installation will fail because it cannot find the trace file.

  9. Rename the %DIVA_HOME%\Program\conf\dfm\dfm.trace.ini file to dfm.trace.

  10. Install the DFM service using the following command:

    %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat install

  11. Start the DFM service using the following command, or open the Windows Services panel and start the service from there.

    %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat start

Contact Oracle Support for more information and details about the full solution and configuration.

Running Multiple Instances of DFM on the Same Computer

Running multiple instances of DFM on the same computer requires a separate installation of DIVA for each additional DFM instance. It is possible to configure only the DFM component in these additional instances. The <serviceName> parameter must be unique among all of the DFM instances. There is no special licensing required to use DFM in this configuration. See Appendix A for Oracle DIVArchive options and licensing information.

The following commands are for administration operations when running multiple DFM instances on the same computer:

Configuring the Service

The DFM configuration file is Program/conf/dfm.conf in each DIVA folder.

Upgrading the Service

You use the standard DIVArchive upgrade procedure to upgrade DFM.

Installing the Service

DFM module is installed with DIVArchive in the same installation package.

The command line syntax to install the service using a specific configuration file is %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat install -conf Program/conf/dfm/dfm.conf.

Uninstalling the Service

The command line syntax to uninstall the service using a specific configuration file is %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat uninstall -conf Program/conf/dfm/dfm.conf.

Starting the Service

You can execute %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat start -conf Program/conf/dfm/dfm.conf to start the service using a specific configuration file from the command line, or you can start it from the Windows Services panel.

Stopping the Service

You can execute %DIVA_HOME%\Program\InterLink\dfm\bin\dfm.bat stop -conf Program/conf/dfm/dfm.conf to stop the service running a specific configuration file from the command line, or you can stop it from the Windows Services panel.

Service Logging

The log folder is %DIVA_HOME%\Program\log\dfm\.

The logging configuration is %DIVA_HOME%\Program\conf\dfm\dfm.trace.

Monitoring the Service

You can use the DFM logs, located in %DIVA_HOME%\Program\InterLink\log\dfm\, to obtain the current status.