Yes you can delete badges, you have 2 options to select from.
Delete but keep existing issued badges - If this is selected then the badge will no longer be available and users will not be able to earn this badge. If a user already has this badge then it will still be displayed on their profile page, however it will be marked as retired (and wont appear on the list of badges on the manage badges page)
Delete and remove all existing issued badges - If this is selected then the badge will no longer be available to any users who have earned this badge and will no longer be able to access it and display it on their profile pages. (by fully deleting a badge means that all the information and criteria records will be removed permanently)