Hmm, I see the Author's question and I don't think anyone, bar suggesting to leave it to his descendants to sort out, has answered it. I think he is not asking how he should have catalogued and directed his collection as he collected it. Answers seem to be on a horse has bolted basis, like me, I think, he wants to know what to do with the dishevelled mass his images and videos have become so far. Duplicate finders are difficult and long winded. I hope there is an answer out there.
PS My answer would be, as I have started to do myself, is to make a title folder, within it, make loads of other folders, all entitled with places, events, people that you know you have encountered over time and then leave it all alone until you feel like tackling the accumulation. Perhaps deciding whether to move an entire folder into your new organised collection or split people, places, events into their new folders. As long as you have a clearly entitled system you can track the position of the various categories and hone the collection with some accuracy. Bit by bit, delete copies and nonsense after you have put them in their places. I have been on this way for ages, it is really quite rewarding to see the mess diminish.....