If you have physical access to a person's computer, then it's easy to install a backdoor. Scripts that will do this for you are readily avaliable for free if you're searching the wrong places on the 'net. You don't even need physical access, just a trusting soul to open your e-mail attatchments. You can get total access to what they do. Doing this does not make you a hacker. It makes you an asshole who uses someone elses script or program to invade another person's privacy.
If someone is keeping remote tabs on a girlfriend's or boyfriend's computer, then that someone has issues.
Hackers? Don't worry. Script Kiddies? Be very concerned. They THINK they're hackers. Keep a firewall up, keep your anti-virus up to date, run Ad-Aware, scan all incoming e-mail (if you have a concience, then you'll have your outgoing automatically scanned as well.), change your passwords regularly, and if you run a file-sharing program or an instant messaging program, ensure that you use the appropriate privacy settings.
mmm... I'm ranting again, but I'm all done for now!