I think it is the unnatural distortions you get when using fisheyes at their widest. IMHO they are just not pleasing to the eye. I get a kind of sickly feeling when I look at them. As if on the deck of a boat in rough sea.
Dont get me wrong I have seen very effective uses of the below 10mm lens but I cant help getting the sense that many people use these lens' as a substitute for creativity.
For example:
This image is, interesting. Architecturally, stunning shapes, vertical and diagonal lines drawing us into the image. Fish eye or no fish eye.. there is interest to be held within.
http://images35.fotki.com/v1154/photos/1/1055548/6080333/DSC_5341dh-vi.jpg
Where as this image relies on the lens to create the interest..
http://images31.fotki.com/v1093/photos/1/1055548/6080333/DSC_5285dh-vi.jpg
Here is an example of what I would call using a fish eye to add interest instead of creating it.
http://www.flickr.com/photos/bazcraig/2082936950/in/set-72157603985719763/