The team at Information Architects have written a wonderful article on icons and their use in UI design.
In that sense, icons are a bit like adjectives and adverbs in regular text. By deleting all of them, your text will not magically turn into a literary masterwork. You need to erase the superfluous ones, find the ones which are needed, and replace the fuzzy and ornate with the precise.
The whole article is great. Oh, and they released a fun game to go along with it as well.