The plain and simple truth is "any way you feel comfortable as long as it is backed up". Where people fail at this is they often consider "backed up" meaning "I save it to an external and delete it off of my computer".
The key is to ALWAYS HAVE 2 COPIES.... that is what "backing up" is.
I would NOT recommend USB backups... USB is the most volitile of media types and most susceptable to data corruption/loss. USB (Flash) storage should always be considered temporary.
Personally, I lease webservers that I host various websites on that I use for my databack up. My hosting service does their own offsite databack up so should anything happen to my live servers, all the information is saved elsewhere and can be restored. I also have two internal harddrives, one for my operating system and programs, one for data. When the drive fills up, I pull it out of my tower, put it on a shelf and buy a new one. I always have at least 3 copies of data, one on my shelf (or in my computer), one on my webserver, and at least one copy on my webservers backup.
Externals seem to be popular, but I just havn't gotten comfortable enough to use them. This is more of an "old school" fear because I'm sure technology has improved, but when I went through all my computer learning, it was always bad practice to move a hard drive. Vibrations could cause the read/write heads to touch the disk surface physically damaging it rendering it a drink coaster. I'm sure modering externals aren't susceptible to this, but I just can't get in that comfort zone.