There are some great ones, and there are some not so great ones. It all depends on the lens. The advantage is usually a HUGE savings over the Canon brand lens when you get into the pro lenses. There is a little bit of compromise but not nearly enough to justify the price difference. I have an arsenal of non canon lenses and I love them. They've allowed me to work with better, faster, sharp lenses that I could not afford to have purchased the canon version of.
Yes, for the most part, the advantage is price. Like most 'premium' brands, you end up paying extra for Canon or Nikon lenses.
When comparing lenses, you can't really compare the whole brand against another, you have to do it on a lens by lens basis. Canon has some pretty terrible lenses and some fantastic lenses. Sigma/Tamron/Tokina have some gems and some dogs as well.
I've used Canon, Sigma, Vivitar and Tamron lenses on my Canon cameras. I would love to have all my lenses be top of the line Canon 'L' lenses, but that is a very expensive goal.
It really is a lens by lens basis here - some of the 3rd party options are poorer, others are just as good as and some are better than. Prices also run the full gauntlet from cheap to very expensive.
You've really got to get into a lens by lens comparison to see what the options are. As an example Sigma make a 50mm f1.4 which beats canons own 50mm f1.4 in pretty much all tests. Further Sigma makes a 150mm macro lens, something canon nor nikon don't make - they also make a 200-500mm f2.8 - the only 500mm f2.8 on the market with a massive price tag attached.
So you gotta get into comparing the options against each other to see which comes out on top. The general view is 3rd party isn't quite as good for less money - but so often that simple assessment is proven wrong in part or in totality.
What kind of budget are you thinking of and also what kind of prime lens? Primes go all the way from wide 8mm all the way up to massive 1200mm options - each with varying uses and price tags.