This parameter determines a file-level prefetching mechanism called zfetch. This mechanism looks at the patterns of reads to files and anticipates on some reads, thereby reducing application wait times. The current behavior suffers from two drawbacks:
Sequential read patterns made of small reads very often hit in the cache. In this case, the current behavior consumes a significant amount of CPU time trying to find the next I/O to issue, whereas performance is governed more by the CPU availability.
The zfetch code has been observed to limit scalability of some loads. CPU profiling can be done by using the lockstat –I command or er_kernel as described here:
Device-level prefetching is disabled when zfs_vdev_cache_size is disabled. This means that tuning vdev cache shift is no longer necessary if zfs_vdev_cache_size is disabled.
0 (enabled) or 1 (disabled)
If the results of er_kernel show significant time in zfetch_* functions, or if lock profiling with lockstat shows contention around zfetch locks, then disabling file level prefetching should be considered.