Focus and zoom are something entirely different.
Zooming will bring objects that are further away closer.
Focus will not change the object size, it will only change the focus point of the camera (the point that is sharp).
The most important thing you should take a look at is the distance you're going to shoot at.
You're placing your camera close to the glass tank where your gecko can get really close. This basically means you're going to need a macro lens.
Macro lenses are able to focus on objects that are really close to the lens.
Most real macro lenses are fixed and can not zoom in so you'll have to zoom by moving your camera forwards. Don't worry, you get used to that really quickly.
I'm afraid a 50mm f/1.8 lens will not be what you're looking for as the closest focus distance on those lenses is about .5m if I remember correctly.
Your gecko is going to get much closer than that! While the 50mm is a great lens you'll really want a macro lens for this.
I myself have the 105mm f/2.8 nikkor which is a great lens. It'll do exactly what you want (and probably more if you get used to working with it).
The 60mm f/2.8 nikkor will probably work just as good, the working distance is just a bit smaller.
I imagine that this difference in working distance isn't a problem with your gecko, it might be a good thing to consider if you think you'd also like to try shooting insects.
You'll really want a longer working distance with insects because they get spooked.
Don't worry about the aperture being 'only' f/2.8 as with macro photography f/2.8 has a crazy small DOF, you'll be shooting at higher apertures like f/8 a lot more because if you don't you simply won't get enough sharpness.
As for the camera: Larger sensor is usually less noise.
This means a DSLR is better than a compact and a FX camera is better than a DX camera.
You will not regret getting a DSLR instead of a compact camera. The quality of your photos will improve loads and it's not too hard getting used to a DSLR, especially with such a fun practice subject.
FX cameras tend to be rather a lot more expensive than DX cameras though and might be slightly overkill anyways.
I myself own the D7000 and am really pleased with the performance. It's ISO performance is simply one of the best when it comes to DX cameras, I can quite easily shoot at 1200 ISO and the images will not be extremely noisy.
I actually shot some images at 2400 ISO by accident yesterday and didn't even notice until I saw the EXIF data.
You can get a D7000 with a 105mm f/2.8 nikkor new for a little less 2000 dollars I guess (not sure off the prices there).