See the section Hardware and Software Requirements in the Sun Java System Application Server Release Notes for specific memory recommendations for each supported operating system.