I don't always ask, but if it's possible I usually do. I've never had anyone say no, but I have had several folks express gratitude that I did ask.
The traditional camera of choice for street photography is the 35mm rangefinder. They are small and very quite. I'd agree with Sash[DSL] that there are some very small P&S cameras (film and digital) that take a very nice photo. My favorite would be the Olympus Epic Stylus (with the fixed f/2.8 35mm lens).
None of the recognized masters of street photography made a habit of asking permission, but they lived in different times. These days with the profusion of small cameras (cell phone cams, etc...), and the internet, folks can get kind of touchy about being photographed. There's this huge group of amateur paparazzi running around trying to photograph embarassing moments, cleavage shots, and up skirt pics for their websites. These guys are making the public skittish.
I was at a wading pool with my daughter. I was watching a grandpa wandering along after his g-daughter snapping away with his digi P&S. He was obviously only taking pics of his kid, but even so, several other parents approached him to specifically ask that he not photograph their children. I thought it was a little weird, but I guess people are afraid of being "exploited".
I'm usually carrying a vintage cam, and folks react different to old cameras. Half the time I think they don't even believe I'm really taking a picture (or that it's even possible to take a picture with an old camera).
The Widelux is an interesting street camera. People are too captivated by the movement of the lens to care that I'm taking their picture, and they don't really understand the 140 degree angle of view I'm getting. Sometimes, when I don't hold it up to my eye, I fire from the hip or even at arms length, and I don't even think that many people recognize it as a camera. I've had some looks of horror like I was using some strange device or possibly setting off a bomb.