This chapter describes parameters that adjust different aspects of the sort memory configuration when using server-side sorting.
This chapter includes the following sections:
Oracle GoldenGate Veridata Server uses virtual memory in the following ways:
Server memory for basic operation. This is the amount of virtual memory that the Veridata server and web components need to operate. It stores object pools, database access libraries, and other information. This is usually about 200 MB.
Sort memory. This is the memory that is used when server-side sorting is used. The virtual memory for sorting is allocated for the entire comparison, not per thread. The rows are read from the agent and submitted to be sorted. The sorting occurs in a thread that is separate from the thread that reads from the agent, and the sort may use more threads to work in parallel. Once all the rows from the agent are submitted to the sort process, the server process retrieves the sorted rows from the sort for comparison.
Row hash queue memory. This is the memory that buffers data between the agent processes, the sort process, and the server process. A comparison that uses database sorting requires a single queue each for the source and target. Each queue has a capacity of 20 MG. The memory usage by the queues is affected by the relative speed of the comparison and by the data coming from the agent. The relative speed between the two agents also affects the memory usage. A larger differential in speed increases the amount of memory that is used, because the queue needs to buffer the data.
MOOS queue memory. This is the memory that holds potentially out-of-sync records between the initial comparison and confirmation steps of a comparison. The size of the MOOS queue is limited to 50K of records. Memory usage is also dependent on the width of each record.
IPC buffer memory. This is the memory that is used to exchange messages between the server and the agent.
Scratch runtime transient memory. This is virtual memory space.
The amount of memory that can be used by the sort process cannot be greater than the minimum of:
System physical memory
Available memory in swap
Java boot option -Xmx maximum memory setting
The maximum amount of memory available to Oracle GoldenGate Veridata is specified by the Java boot option -Xmx. When server-side sorting is used, a large portion of this memory is reserved for sorting during comparisons. This reserved amount is controlled by the server.max_sort_memory configuration parameter.
When a comparison is run, two buffers are allocated from the reserved sort memory. Each of these is equal to the size specified as Maximum Memory Usage (MB). To access this setting click the Edit option from the Profile Configuration screen, then Sorting Method from the Profile settings categories.
To Estimate Memory based on the Number of Concurrent Comparisons
The maximum amount of memory that can be used for any comparison is set by the parameter server.max_comparison_sort_memory. The -Xmx Java boot option should be set large enough to allow the desired number of concurrent comparisons.
The maximum number of concurrent comparisons is defined by the server.max.concurrent_comparison_threads configuration parameter. Therefore the maximum amount of sort memory can be as large as:
server.max_comparison_sort_memory * server_max_comparison_threads
For example, if you set server.max_concurrent_comparison_threads to allow 10 concurrent comparisons and leave server.max_comparison_sort_memory set to the default value of 100 MB, you will need 1 GB of available memory.
To Estimate the Amount of Memory Used per Row
Refer the section "Disk and Memory Requirements for the Server Component" in Installing and Configuring Oracle GoldenGate Veridata for the calculation to estimate the amount of memory used per row.
To set a parameter, edit its entry in the veridata.cfg file. This file is stored in the DOMAIN_HOME/config/veridata directory within the Oracle GoldenGate Veridata Server installation directory.
Open an Oracle service request before changing these parameters. For more information, go to http://support.oracle.com.
This section describes the parameters that can be set in the veridata.cfg file. These parameters are grouped under the following categories:
This section defines the following configurable parameters for your Veridata Server:
The directory that contains Oracle GoldenGate Veridata reports.
server.veridata_data path
where path is a relative or absolute path for the directory where Veridata reports will be stored.
Note:
If you specify a relative path for the data directory, you need not start the path with a forward (/) or backward (\) slash. The path will be relative to the Veridata domain home directory.veridata/reports
That means the default data directory is VERIDATA_DOMAIN_HOME/veridata/reports.
This parameter defines the persistence database type.
server.veridata_data database_type
where database_type is the persistence database type. The options are:
ORACLE_OCI
MS_SQL
MYSQL
ORACLE_OCI
This parameter defines the meta-session handle timeout in seconds.
server.meta_session_handle_timeout seconds
server.meta_session_handle_timeout 600
900
This parameter specifies the maximum number of jobs that can be run simultaneously.
server.max_concurrent_jobs number_of_jobs
server.max_concurrent_jobs 200
100
Sets the maximum number of concurrent comparisons that can be executed. In general, the amount configured by the server is the optimal value, given the machines resources. You can lower this number to reduce the impact of the server on your system. When this limit is reached, no new comparisons will start until an active comparison completes.
server.max_concurrent_comparison_threads {default | number}
default allows Oracle GoldenGate Veridata to compute the maximum number of concurrent threads based on the server.max_comparison_sort_memory and available resources. The default value is the server.max_sort_memory divided by server.max_comparison_sort_memory.
number is a positive integer that sets the maximum number of concurrent comparison threads.
server.max_concurrent_comparison_threads 100
The default value is based on the available virtual memory.
Sets the maximum amount of sort virtual memory that is available to all running comparisons that use server-side sorting. The default amount is the Java boot option -Xmx maximum memory setting less the 200 MB needed for basic tasks. You can limit this amount to make more memory available for the Oracle GoldenGate Web User Interface.
If a comparison cannot get enough virtual memory when it starts, it fails or tries again depending on the setting of the server.sort_waiting_threshold parameter. If a comparison does get enough virtual memory, the currently available sort virtual memory gets decremented by the amount that the comparison reserves. When a comparison completes, it increments the amount of available sort virtual memory by the amount of sort virtual memory that it had reserved.
server.max_sort_memory {default | number{M | m}}
default allows Oracle GoldenGate Veridata to define a maximum value that is dependent on the operating system.
number{M | m} specifies a value in megabytes. For example, 1000M means a limit of 1000 megabytes. If this number exceeds the amount of available memory, the value will be reduced to the amount of available memory.
server.max_sort_memory 1000M
The system calculates the default size based on the available virtual memory.
Sets the maximum amount of virtual memory that can be reserved by a single comparison that is using server sorting. A comparison that tries to reserve more than this amount will fail, even if it is configured to retry.
The amount set for this property limits the amount that can be set in the Oracle GoldenGate Web User Interface Profile.
If the value of this parameter is changed, it might cause the concurrent comparisons that use server-side sorting to fail or retry, depending on the sorting parameters defined in the profile that is being used.
server.max_comparison_sort_memory {default | number{ M | m }}
default is the value of server.max_sort_memory.
number{M | m} specifies a value in megabytes. For example, 600M sets the limit for any comparison to be 600 megabytes.
server.max_comparison_sort_memory 1000M
The default value is the value of server.max_sort_memory.
Sets a threshold above which new jobs will fail if there are too many comparisons waiting to retry a virtual memory reservation.
server.sort_waiting_threshold {default | number}
default is zero; if memory cannot be allocated the job will not be allowed to run.
number sets the number of comparisons that can be waiting before new jobs will fail. For example, a value of 100 means that if there are 100 comparisons waiting to retry a memory reservation, and there is a job that is ready to begin execution, this job will not be allowed to run.
server.sort_waiting_threshold 1000M
The system calculates the value.
This parameter specifies the number of writer threads per sort directory.
server.concurent.writers number
server.concurent.writers number
The default value is 4.
This parameter specifies the number of reader threads for the entire server.
server.concurent.readers number
server.concurent.readers number
The default value is 4.
This parameter specifies the number of threads used to sort input buffers from the Veridata Agent.
Note:
The value ofserver.number_sort_threads should not be greater than the number of available processes.
server.number_sort_threads number
server.number_sort_threads number
The default value is 2.
This section defines the parameters that you can use to configure SSL communication between your Veridata Server and Veridata Agents:
This parameter specifies whether SSL is enabled for communication between the Veridata Server and all Veridata Agents.
server.useSsl [true|false]
server.useSsl true
The default value is false.
If the value of this parameter is set to true, Veridata Server allows SSL communication between the agent and the server when a trusted certificate expires.
Note:
The parameter is not applicable if you are running IBM's JVM.server.ssl.client.allowTrustedExpiredCertificates [true|false]
server.ssl.client.allowTrustedExpiredCertificates false
The default value is true.
This parameter specifies a name for the identity store key factory algorithm used for SSL communication.
server.ssl.client.identitystore.keyfactory.alg.name=algorithm_name
server.ssl.client.identitystore.keyfactory.alg.name=IbmX509
If you are running on IBM's JVM, set the value to IbmX509.
The default value is SunX509.
This parameter specifies a name for the trust store key factory algorithm used for SSL communication.
server.ssl.client.truststore.keyfactory.alg.name=algorithm_name
server.ssl.client.truststore.keyfactory.alg.name=IbmX509
If you are running on IBM's JVM, set the value to IbmX509.
The default value is SunX509.
This section defines the following configurable parameters for your Veridata Server:
To run the Veridata command-line utility from the Veridata Managed Server, set this parameter value to true.
veridata.cli.run_from_managed_server [true|false]
veridata.cli.run_from_managed_server false
The default value is true.
This parameter specifies the name of the Veridata Managed Server.
veridata.cli.managed_server_name server
veridata.cli.managed_server_name VERIDATA_server2
The default name of the managed server is VERIDATA_server1.
This parameter specifies the listening address of the host machine for the Veridata Managed Server.
veridata.cli.server.listenAddress host
veridata.cli.server.listenAddress host.example.com
The default name of the managed server is localhost.
This section defines the configurable parameters used for report file encryption:
When this parameter is set to true, the comparison report artifacts will be encrypted. Otherwise, the report contents will be in clear text.
server.encryption=[true|false]
server.encryption=false
The default value is false.
This parameter specifies the strength of the encryption algorithm. Valid values are 128, 192, and 256. If set to a value other than 128, you must install Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files.
server.encryption.bits=[128|192|256]
server.encryption.bits=128
The default value is 128.