I think it's all been said above, but because it took me a bit to figure it out when I started, let me summarize.
In short, the human brain, via the eye, is naturally drawn to the brightest color in the frame or, in the case of black and white, the brightest white area. Like it or not, it's just the way we are wired. Unless you happen to have some other great point of contrast, that's where the eye will go. In your first shot, the brightest color is the stack of chairs and the brightest white in the b&w portion is the door handles, neither of which are the focal point.
You've fixed the color issue, now go back with the burnish tool set for highlights (in photoshop anyway) and darken the handles. You don't need to go anywhere near black, but tone them down.