Can I create a custom validation rule for a field?

Yes, you can write a validation rule to display an error if the rule you specify isn't met.

Let's say you want the D-U-N-S number in the UI show a validation error if someone enters a D-U-N-S number that's already in the system. In Application Composer, write a validation rule to display an error if a duplicate gets entered.

See Server Scripts for more information. DUNS