This is just my own personal opinion, however these days I strongly suspect it has as much to do with "marketing" as anything else. In the earliest days, it may have been about manufacturing costs, but it's been my experience that in cases such as this, "consumer price" seldom reflects actual manufacturing costs. While -no-, I don't work for any camera manufacturer to know for sure, in this case I doubt the actual manufacturing costs between the APS-C sensor and FF are THAT different. I'm sure there -IS- a difference, however I'd be REALLY surprised if someone were able to prove they were significant enough to warrant 2 to 4 times (or more) the purchase price of the respective camera bodies that employ such technologies.
This is just a guess on my part, but I'd suspect that today it's simply a matter of consumer demand more than anything. It's tempting to say that because something like a Nikon D4X cost $6000, that it must be considerably more expensive to produce than a D5500, however manufacturing and mass produced products seldom work that way (comparatively speaking). -IF- were were talking about components that were all hand made, that might be the case, however in what's essentially an assembly line, mass production environment, things change dramatically, which suggests there MUST be some other dominant factor involving cost. Consider this; the folks at Nikon, Canon, Olympus, etc., KNOW there's only a limited number of truly professional photographers out there willing to spend $2000 - $4000 (or more) on a "pro body". Sure there's a few "non-pro" folks out there who will dump the money...if nothing else, just because they can, however I suspect that percentage is comparatively quite small and that the camera makers realize this. What's more is I'm sure they also realize that most people just don't need ALL the features such cameras offer...after all, "Aunt Jane" is unlikely to have the same needs in a camera as the local portrait studio. What's more is that I'm sure they know that MANY average consumers just don't have THAT kind of coin to blow either...if those $6000 bodies were ALL the camera makers had to offer, I suspect A LOT of people would just go back to film (I know I certainly would!). When it comes to marketing, I'm sure they're also painfully aware of other factors such as used cameras, gray market, yadda, yadda, yadda...all of which impact new camera sales. The thing to remember is that companies like Nikon, Canon, etc., are "businesses"...in the end game, they're all about profit and how to maximize it.
With this in mind, for large companies such as Nikon and Canon (and others), despite manufacturing costs, it makes a great deal more sense to provide various camera styles, at various price points to cover the widest range of consumers possible. You can make those cute and stylish little compacts around the $100 - $300 mark that work GREAT for people like Aunt Jane who really has no use at all for any benefits regarding full frame and probably wouldn't even know where the power button is on an average DSLR (no offense Aunt Jane, LOL). They may not make AS MUCH profit as they do with say, that $6000 pro model, but they do make SOME profit and they certainly make up for the difference at other levels. Then you put out the entry level DSLR's and the "prosumer" models ranging from $600 - $1500 for those who take their pictures a little more seriously (or simply have a few more bucks to blow)....I don't have the statistics, however I suspect this is very likely their most popular market...from a profit point of view, I'd suspect this is really their bread and butter. Mid range products for the "middle class" and all. And then you have the top shelf stuff for folks who can write it off on their taxes...you get the idea. Essentially it's the all mighty corporate attempt to be "all things to all people" and for the most part, I suspect it works very well. After all, if Aunt Jane is happy with that little Nikon "Coolpix", she -may- be more likely to go with a Nikon D5500 the next time she buys a camera.
Again I have no specific facts or statistics to back that up, but it seems to make a great deal more sense than just assuming the expensive cameras are more expensive to make...large manufacturers, cameras or otherwise, don't typically work that way as a general rule.