Because you are 'magnifying (narrowing the FOV) the causes a magnifying of any movement as well. In other words your shake is amplified proportionate to the focal length. remember the old adage.saw. rule of thumb about shutter speed should be at least 1/focal length? Well that;s Why you can shoot a 50mm at about 1/40 of a sec with no movement blur. Now apply that to 600mm...you get 1/600. And on a crop frame camera your better off with 1/1000. My experience has been that even with the better lens image stabilization is not as effective above about the 500mm range. Maybe me, but that is what I have found.
Personally I think that the rule of thumb works between around 50mm and 100-200mm ish. Beyond those limits things change a bit. Shorter focal lengths magnify the image much less thus you can often handhold wide angle shots at much slower shutter speeds than the rule would suggest possible. Similarly, for many people, the opposite is true with longer telephoto lenses. Partly its an issue of strength, fatigue and weight as those bigger lenses are a lot heavier; but also the magnification increases and thus the potential handholding speeds go up.
Of course experience and practice can change these values too so they are only ever a guide-line to give you a starting point.