If you don't care about size and weight and 400mm is enough, and or 560mm with a 1.4 then get the 400mm.
I have a 500mm vr and love it, because i do birds a lot and would rather have the distance over a faster lens. The 500mm is also right on the edge of whats still comfortable for hand holding. I have shot from a car while in the driver seat, from a kayak, and carry it around on a black rapid strap for hours, without too much discomfort. Also using proper bracing techniques i have been able to hand hold it without fatigue for longer then needed to get photos.
I have a 400mm af-i 2.8 im working on fixing, its a 14lb beast, so it will probably only get used for fun. There is really something special about that 2.8 and 400mm, kinda like f4 and 600mm, just makes images stand out so much more because of shallow DOF and outstanding sharpness of the 400mm