transactionlimit limits the number of messages allowed per connection. This can be used to thwart attackers in the following way:
An attacker can connect via SMTP and send many RCPT TO commands in an attempt to guess legitimate email addresses. Such an attack can be thwarted by limiting the number of invalid RCPT TOs allowed in a transaction. The attacker may respond by using multiple transactions, but with transactionlimit you can limit the number of transaction allowed in an SMTP session. The attacker can use multiple sessions, but now his cost is getting prohibitive. Connection throttling can be used to limit the number of sessions in various ways making the cost really prohibitive in most cases.
This is not without cost our side, however. Some SMTP clients react badly to recipient limits, transaction limits, or both. Exceptions need to be made for these clients. But TCP channel options apply to the SMTP server unconditionally. The solution is to use channel keywords and switchchannel to route problematic agents to channels with larger limits.