This screenshot shows the location of the delete button above the list of managed servers.