So, in an overly simplified explanation.
Computers do everything in terms of math -- that transform in photoshop is just a series of addition subtraction multiplication and division by the computer.
32 bit numbers are 32 digits long (in binary)
64 bit numbers are 64 digits long (in binary)
It's faster for your computer to make 1 math operation on two 64 bit numbers then 2 operations on four 32 bit numbers (this is a very simplified explanation)
In addition, as Garbz mentioned, 64 bit computers can use more RAM (32 bit processors have a 4GB limit).
So if you have a choice of the 32 bit or 64 bit version, use the 64 bit version.
But you need to have a 64 bit motherboard and processor combo to make that work. If you don't have that, the 64 bit version will refuse to run.
You'd have to look at your computers specs to determine if it's 64 bit
Also, you may see in some programs the option to use GPU / use graphics card. Basically your graphics card / GPU, is an additional processor, and some image editing programs can take advantage of it to speed up processing