To prevent hosts that reside outside your domain from relaying to other hosts outside your domain:
Split incoming mail into different channels. For example:
IP addresses within your domain go to the tcp_internal channel.
Authenticated sessions go to the tcp_auth channel.
All other mail is sent to the tcp_local channel.
Recognize and allow mail from your POP and IMAP clients by using an INTERNAL_IP mapping table, fully explained in the chapter on Mail Filtering and Access Control in the Sun Java System Messaging Server 6.3 Administration Guide.