Yes, if you are limited by how far back you can get, shooting vertical will allow you to get more height...but the main reason for shooting stitched panos in vertical, is that because the long edges of the image are closer to the centre, they have less error/distortion. This is especially important if you're using a wider angle lens....where the edge distortion can get pretty bad.
The most common recommendation I've heard, is to overlap each shot by 30%.
Just like most thing in photography, there are quick and dirty techniques that may be 'good enough', but there are tools and techniques to take it as far as you could want.
There are point & shoot cameras where you just set it to pano mode and sweep it across the scene, it creates the pano right in-camera.
On the other end of the spectrum, for best results, you will want to rotate the camera around it's nodal point...but with most tripods, if you turn the camera to a vertical orientation, you put the camera way off centre. Not only would you not be close to the nodal point, but you put the camera in a much less stable position on the tripod. The solution is to get an L-bracket, which allows you to secure the camera to the tripod head in both vertical or horizontal, keeping it centred. There are also special tripod heads (or accessories) that allow you to tweak the position of the camera on the head, allowing for more precisely aligning the nodal point to the centre of rotation. This is more important if your shot will have both near and far aspects.
Another thing that can be rather important, is that the rotation be as level as possible. That's easy enough if you're on flat ground, but if your tripod legs & base, aren't level, and you use the head to level the camera, the rotation probably won't be level as you go from shot to shot. This is where photographers use something called a leveling plate. It's a device that goes between the tripod and the head (sometimes built into the tripod). It allows you a limited degree of movement, just enough to level the top surface of the tripod, so that the tripod head itself if level, not just the camera.
And of course, if you're shooting for the best possible image quality you can get, you'll want a tripod that is very stable....and will be able to limit vibration of the camera from things like mirror and shutter movement.
Many will say that this is all overkill....but you probably won't hear that from a photographer who makes their living selling landscape photos.
Lastly, there are robotic devices that will automatically move the camera in precise increments so as to give you images for stitching. I'd think that theses are more commonly used for creating multiple row pano stitches. So rather than taking 5 shots in a row, you could take two rows of 5 shots each, either giving you a taller pano or allowing you to use a longer focal length, like with less distortion. Either way, it allows you to multiply the number of pixels in the final stitched image.