To optimize preformance, observe the following guidelines when setting up Oracle 6.4 TB NVMe SSD v2s in a server.
Block size can be configured through a server operating system or file system and is set to a default size with Oracle databases.
Oracle 6.4 TB NVMe SSD v2s are designed to provide best performance for data transfers that are multiples of 4k size and using addresses that are 4k aligned. Partitions should be aligned to start on 4k boundaries.
The ZFS file system might require manual alignment. Oracle 6.4 TB NVMe SSD v2 has a maximum transfer size of 128k. IO requests for larger transfer sizes are broken into transfer sizes of 128k or smaller. For optimal performance, transfer sizes should be limited to 128k to avoid additional overhead associated with breaking into smaller transfer sizes.
Oracle 6.4 TB NVMe SSD v2 should be formatted using a label of type EFI (format -e command).
Ensure that when the EFI label is created by ZFS, the default start sector is 256, which aligns S1 with 128k (if the block size is 512). The vtoc label default cylinder size is 50176 (224*224) blocks. If the block size is 512, the default Oracle Solaris operating system partition aligns with 512k. For example: 50176*512 = 49*512*1024.
Specify and ensure 4k alignment: The default start sector of 34 for EFI labels is not a 4k aligned value. Use the partition subcommand of the Oracle Solaris format command to change the start sector to 256, or any other 128k aligned value. Note that there are 512B per sector.
The ZFS file system automatically aligns partitions to start on 8k boundaries when a full disk is allocated to ZFS (recommended). If you allocate individual EFI partitions to a ZFS pool, ensure that the partition is 4k-aligned.
For highest performance, ensure that the system meets the physical, environmental, and electrical specifications listed in Specifications.
Tuning ZFS When Using Flash Storage https://docs.oracle.com/cd/E53394_01/html/E54818/chapterzfs-flash.html