This chapter describes an overview of the DFM (Drop Folder Monitor) component of the DIVArchive system and includes the following information:
Drop Folder Monitoring allows users and third party applications to deliver content to be archived by copying related files to a folder, an FTP server, or a CIFS share.
Note:
Linux-based Actors do not support UNC paths for CIFS sources and destinations.DFM does not require an application, or user, to send the Archive command. Each time a file (Single File Mode), or set of files (File Set Mode), is placed in a monitored Drop Folder, DIVArchive automatically archives the files and creates the related DIVArchive object according to the rules set in the configuration file for that particular Drop Folder.
The flexibility of DFM enables a myriad of possible configuration combinations. Oracle has combined the most commonly used configurations into four basic DFM Solutions to make configuration and operations easier. If necessary, you can perform advanced DFM configuration to employ DFM beyond the four solutions described in this book.
DFM supports multiple automated operations. Example configurations have been included to aid in the configuration of specific workflows as follows:
dfm.conf.delete.iniThis configuration file contains the minimum configuration parameters to configure DFM for use as a file deletion solution.
dfm.conf.single.local.iniThis configuration file is a template for configuring DFM for automated local, single file based archive operations.
dfm.conf.iniThis configuration file is a template demonstrating how multiple workflows and Drop Folders can coexist in a single configuration file, and is modular by design.
Drop Folder Monitor is a component of Oracle DIVArchive that can be installed with the DIVArchive installation package.
DFM monitors designated Drop Folders and requests DIVArchive to archive objects found in these folders. Drop Folder Monitor can monitor up to 20 local folders, FTP folders, and CIFS folders; each with its own individual configuration. There are two types (modes) of Drop Folders; Single File Mode and File Set Mode.
For example, if the archive contains two media groups, News and Movies, then you must configure two Drop Folders; one for News and one for Movies. However, if File Set Mode is used (or an advanced configuration setup), you can use a single Drop Folder instead.
After archiving completes, the files are deleted from the folder they have been archived from.
Best practices dictate that files should be copied to the Drop Folder and not moved. After the configured time period, if the file is considered incomplete, it might be deleted. Copying the file, rather than moving it, ensures that a backup copy is available if an unwanted deletion occurs.
When the Drop Folder is configured on an FTP server, you must copy all content (files and folders to be archived) to the Drop Folder using FTP. If you copy the content to the Drop Folder using another process (local copy through the operating system, remote desktop copy, and so on), there is no guarantee DFM will process the content properly.
You use Single File Mode Drop Folders to automatically archive objects containing a single file. This mode is convenient for MXF, GXF, and so on, with single file assets. There can be many single files in the folder, but each will only be archived as a single file object.
The Object Name is the file name including the extension. The Object Category, priority, and other archive command parameters are defined in a Drop Folder configuration file (dfm.conf). The configuration file contains the description of all folders to be monitored by DFM.
When a file appears in a configured Drop Folder, DFM monitors the file size. If the size has not changed after a preconfigured time threshold (last modification time), then the file is considered complete, and the archive is initiated. After the file is archived it is automatically deleted from the Drop Folder.
Caution:
The last file created in the File Set Drop Folder must be the metadata file (with the.mdf extension). The presence of this file triggers the archive operation. If the metadata file is transferred before the actual data files, content loss can occur!You use File Set Drop Folders to automatically archive objects containing more than one file, and is compatible with complex objects.
When monitoring a File Set Drop Folder, DFM checks for the presence of a metadata file (.mdf). This file describes the details of the content to be archived, and signals DFM that the content to be archived is ready. The archive process starts only after this file is available to DFM.
Using the metadata file allows specifying the Object Name, Category and user Comments on a per object basis.
You (or an application) must first create a folder within the Drop Folder for the set of files intended to be archived as a single object in DIVArchive. The folder must have the -allow_delete_on_source and -r options configured. After the folder is created, the data files have to be copied to the folder.
The object archiving process is initiated after DFM has opened and parsed the metadata file. After the object is successfully archived, DFM deletes the folder, and all of the files within it, including the .mdf file.
Oracle offers the four most common DFM Solutions as described in the following sections. Select the solution, or solutions, that meet your requirements, and follow the configuration instructions for those particular solutions.
See Chapter 2 for details and basic configuration of each solution.
This DFM solution can monitor local disks, FTP transfers, and CIFS folders. DIVArchive will archive each file found in the Drop Folder as a single object. If your requirements include only single files then this solution will work perfectly.
Only single file objects are supported.
Supported major configuration parameters include:
Oracle DIVArchive Manager
Drop Folder URL
Category Name of objects being archived
Source/Destination for the Archive request
Templates for File Path Root and File Name generation
DIVArchive Media Name for the Archive request
If the connection to the disk, FTP or CIFS is unavailable (or disconnected) for a period (that is, when the fileReadyThreshold time has elapsed), the file will be considered complete when it is actually incomplete.
You will only find the fileReadyThreshold parameter in the dfm.conf.ini, and not in the dfm.single.conf.ini configuration file. However, you can manually add this parameter to the dfm.single.conf.ini file if wanted, or necessary.
This DFM Solution offers more flexibility than the Basic Solution. This configuration can monitor local disks, FTP servers, and unsecured CIFS folders containing either single files or file sets. Secured CIFS can also be used if you run the service under a suitable user account.
File Sets include a metadata file containing the details of the files included in the set. Each metadata file found in the Drop Folder is parsed and a new object (usually with more than one component) is archived in the DIVArchive system.
Single and multiple file objects are supported.
Recursive Archive (folder and files) is supported.
Supported major configuration parameters include:
Oracle DIVArchive Manager
Drop Folder URL
DIVArchive Media Name for the Archive request
This DFM solution monitors only the local disk, and removes files that are obsolete. If a subfolder is empty for more than the configured allowed time period, it will also be removed. This solution is used with the other DFM solutions, and can be included in all DFM configurations.
Supported major configuration parameters include:
Folder URL
Interval specifying when the file will be deleted from the folder.
This DFM Solution monitors only the local disk and can use either Single Files or File Sets. Each Metadata File found in the Drop Folder is parsed and a new object (usually with more than one component) is archived in the DIVArchive System.
See Appendix A for Oracle DIVArchive options and licensing information.
This solution is used with Oracle DIVAnet systems that are configured with inter-site transfer capabilities. Contact your Oracle Sales Support Specialist for detailed information on various DIVAnet Solutions available.
Single and multiple file objects are supported
Recursive Archive (folders and files) is supported
Supported major configuration parameters include:
Oracle DIVArchive Manager
Drop Folder URL
DIVArchive Media Name Pattern for generation of the Media Name for the Archive request
Source/Destination Name
Source/Destination Path