Can I reassign indirect reports of the manager from the same Direct Reports task?

No, you can't. You can reassign only direct reports of the manager.