The matter may have already been decided, but for what its worth, I have shot with an E-510 for the last year, and have an OM 50mm f/1.8. If you are using the viewfinder to manually focus, then you are wasting your time at f/1.8 or f/2.8.
However, Olympus has the wonderful option of live view: compose your image, select your point of focus, zoom in 7x or 10x, and fine tune, then release the shutter. Once you practice this technique, you can achieve some fabulous shots with the OM glass. You can check my flickr photostream for proof of that.
The E-1, E-30, and E-3 all have much larger viewfinders comparable in magnification, coverage and brightness to film SLRs, in which the OM glass is much easier to manually focus.
If manual lenses arent your thing, then like others have suggested, the ZD 50mm f/2 is an excellent choice. That lens is razor sharp, fast, weatherproof, does 1:2 macros, and is relatively cheap for what it does. If you are considering a different body in another brand, you might better use that money on an excellent lens! If you still decided to switch, you would be able to sell the ZD 50 for most of its value very easily.
I am getting long winded, so I will end with this: any camera is only as good as the glass you put in front of it, I am sure users of all brands can agree on that...