How can I restrict certain file formats from being uploaded?

You can use the Manage Attachment File Formats setup task in the Setup and Maintenance work area. Using this task, you can configure the file formats that you want to restrict from being attached.