System Considerations

Review the following system considerations for Direct Transfer:

  • Memory usage on PeopleSoft

  • Memory usage on OpenSearch

Memory Usage on PeopleSoft

The amount of memory used by Direct Transfer for runtime data storage is dependent on the segment size and number of attachment handlers. For example, for the default segment size of 10MB and 20 attachment handlers, Direct Transfer would utilize an additional 200 MB on the PeopleSoft Batch Server during data send.

Memory Usage on OpenSearch

When Direct Transfer sends data to OpenSearch, review the main memory considerations on OpenSearch. This memory allocation should be configured using the JVM.option configuration file in OpenSearch.

  • The incoming data is stored in bulk thread queues on OpenSearch during the ingestion process. The amount of memory used for this purpose is based on the bulk thread queue size.

  • The amount of parallel ingestions which can happen on OpenSearch is based on the cores on the system. The number of parallel bulk ingestions is equal to the number of cores.

  • During ingestion, for documents containing attachments, OpenSearch uses a large amount of memory for document parsing. An example is parse a large attachment of 10 MB size, the memory required for parsing it is around 100 MB.

Example 10-1 Examples

  1. To index a search definition where the average attachment size is 100 KB and OpenSearch server is a 2 core system and memory available for OpenSearch JVM is greater than or equal to 8 GB, then an Attachment Handler value of 20 should suit in most circumstances.

  2. To index a search definition where average attachment size is 1 MB and OpenSearch server is a 4 core system and memory available for OpenSearch JVM is greater than or equal to 16 GB, then an Attachment Handler value of 10 should suit in most circumstances.

  3. To index a search definition where average attachment size is 100 MB and OpenSearch server is a 4 core system and memory available for OpenSearch JVM is greater than or equal to 16 GB, then an Attachment Handler value of 5 should suit in most circumstances. The http.max_content_length setting on the opensearch.yml configuration file should also be increased in this scenario. The default value is 100 MB, but for large attachments you may set the value to a higher value, for example, http.max_content_length=512mb