Soft baby ---> soft light.
I agree, using a nice soft light for babies is usually best. To make a light soft(er), you need to make it bigger and/or get it closer to the subject. So a large softbox or a large umbrella, close to the subject, would be best.
Softbox vs Umbrella....they both soften the light and the bigger they are, the softer the light can be. It's not really a matter of better or worse. I like softboxes because they constrain the light to one direction, so that it's not bouncing all around the room. This is very helpful when going for a dark background, but not as much when going for a lighter background.
Most 'good' softboxes have an inner layer of diffusion, so that they give a nice even light. Cheaper softboxes or some umbrellas will have a hot spot in the middle, so that your lights isn't nice and even.
I don't know that you have to go to the trouble of trying to use two umbrellas on one light. And I'm not sold that using a reflector is going to be any better than using the two lights for main & fill. (unless you need one of those lights for something else, light a background light).
Natural light can be a nice option, especially because a big window can give you really soft light. But then you are subject to the shutter speed and ensuring that it's fast enough to give you sharp shots. With strobes, you don't really have to worry about that.