The Identity Provider does not have to be an OpenSSO Enterprise deployment. But for optimum protocol interoperability, use OpenSSO Enterprise. See the Sun OpenSSO Enterprise 8.0 Installation and Configuration Guidefor detailed installation and configuration steps.
A good practice is to install the Identity Provider and Server Provider in different domains. If that is not possible, make sure they use different cookie names or cookie domains. This eliminates cookie validation inconsistency.