On a serious note, the rubber ducky was a great idea. You know how if you give a kid something to play with during a diaper change, it goes a lot quicker? Same idea, give him something to keep his hands busy. You'll have more luck keeping him in one spot, and props can always make a photo more interesting.
Yup young kids can be tricky to keep still lol. Have the same with our 3 1/2 year old. Like the black and whites. Not fond of the desaturated colors some how it makes it look muddy. 4 and 5 look a bit soft. Next time put some distance between him and the back ground will help the shadows. You could try a diffuser for your pop up if your using a point and shoot to soften the light.
Lighting is pretty harsh...a couple look pretty oof...rubber ducky pic is the best imo....use the clone tool to fix any wrinkles in your backdrop...and duct tape to keep him still..haha nah, he's a boy, he'll break right out of the tape! Keep at it...practice, practice, practice! P.S. Cute lil drool bug ya have
It's good your starting off taking pics of him so early with a smaller point and shoot. My son got stuck looking at a DSLR very young and at 3 1/2, he says "no camera mama", huffs and puffs, and runs in the other direction. If you think your son moves a lot now, just you wait. You've seen nothing...yet
Get rid of the background, turn off the flash, open the windows and let in the natural light. at the very least I would move the background away from him. Your viewpoint also tells nothing more than "mom holding a P&S camera".
Bring in the natural light, set the baby in a "baby setting" and give him something to play with, lay down on the floor and interact, looking for reactions. If you can control the aperture on the camera, set it to the lowest number.