Yes fixed (or nearly fixed*) focal length lenses are called prime lenses. 'Prime lens' has another meaning as well: the main lens in a combination of lenses, eg a prime plus a close-up lens, or a prime plus a teleconverter.
As you say, fixed focal length lenses have an aperture advantage over zooms. The fastest readily available zooms for full-frame 35 mm (format matters) are usually f/2.8, while f/1.4 and even f/1 primes are available. That two or three stop difference can be a great advantage.
Primes are also lighter - an f/1.4 50 mm is lighter and smaller than, say, a 25-50 mm f/4 and a 35 mm f/1.4 is much smaller and lighter than a 17-35 mm f/2.8. It is easier to design a prime with an optical performance that surpasses that of a zoom, though if you pay enough for a zoom you can match the performance of a prime.
I use primes a lot. I would far rather use two cameras, one with a 35 mm and one with a 70 mm prime, than a single camera with a zoom.
*There are primes that change their focal length as they focus. Most of the current Micro-Nikkor primes shorten their focal length as they are focused more closely, so that they can achieve very close focus without excessive extension. Motion picture lenses do similar thing so that the angle of view stays the same as the focus is changed - otherwise a focus pull would look like a zoom.
Best,
Helen