Removing a fan module is a hot-swap operation. You do not need to power off the switch before you remove the fan module.
See Install a Fan Module.