Archival Testing

About Archival Process

Archival testing refers to the process of testing and validating the archival and retrieval functionalities of a system, software, or data storage solution. This type of testing is particularly important for systems that handle large volumes of data meant for long-term storage, such as archival systems, databases. The goal of archival testing is to ensure that data can be stored, retrieved, and maintained accurately over an extended period.

Performance Testing

Objective: To validate performance-related issues and identify defects for optimization by assessing the system's performance with large volumes of data while archiving.

Hardware Specs

	Source Database (Oracle DB 12c)	Target Database (Oracle DB 12c)
Attached Vnic(s)	primary	primary
Block Volume	5346(GB) : ATTACHED	2258 (GB) : ATTACHED
Boot Volume	100 (GB) : ATTACHED	100 (GB) : ATTACHED
Burstable	BASELINE_1_2	BASELINE_1_2
HW	2.55 GHz AMD EPYC 7J13 (Milan)	2.55 GHz AMD EPYC 7J13 (Milan)
Memory	384	384
NetworkType	PARAVIRTUALIZED	PARAVIRTUALIZED
OCPU	24	24
OS version	Oracle Linux Server 8.6	Oracle Linux Server 8.6
Shape	VM.Standard.E4.Flex	VM.Standard.E4.Flex

Archival Settings in application.yaml file

batchSize	1000
threadpoolSize	20

 

Test Results

Through performance testing, we experimented with batch size and thread pool configurations. Our findings revealed that with a batch size of 1000 and a thread pool size of 20, we achieved an impressive throughput of 47,522 activities per minute i.e., we are able to transfer ~6GB of data.

CPU Utilization of Source DB:

CPU Utilization of Target DB:

Indexes proposed during this phase of performance testing:

CREATE INDEX "IX1_ARCHIVE_BATCH_ENTITYGUIDS" ON "ARCHIVE_BATCH_ENTITYGUIDS" ("ARCHIVEGUID","BATCHID") ;

CREATE INDEX "IX2_ARCHIVE_BATCH_ENTITYGUIDS" ON "ARCHIVE_BATCH_ENTITYGUIDS" ("ENTITYGUID");