depends on the filter, i keep my UV filter on 24-7 as a lens protector
and i keep my polarizer on every time i shoot outdoors, but take it off when shooting indoors unless i need to cut down reflections
really its about what filter you use and when you plan on using it