Can I create a custom alert or notification when exceeding user-defined limits in Budgetary Control?

No, there isn't built-in functionality to support this.

An alternative solution would be to create a report that tracks budget accounts under the defined threshold and use the OTBI agent to provide event-driven alerting. You can define a schedule to run the agent, a data condition that determines what the agent does, and the analysis that can be distributed.