Yes, it's safe. I changed lenses inside and outside on film SLRs and rangefinders, now I do it on digital ones too... just do it away from strong wind and water (and, I guess, a sandstorm), try not to leave the lens off for very long, and nothing harmful should get in there.
At what frequency do you need to clean the filter?