Manual focus issues with rx10 m4

[ralph isaacs]

I'm using manual focus for dark sky photography without a telescope. I've turned off manual focus assist as it is inappropriate. The scale that appears when adjusting the focus ring is wildly out of calibtration. For instance if set to infinity, a bright star appears doubled. I'm resorting to taking an illuminated resolution target with me, focussing on that in a high telemode. Then I tape down the focus ring and go to lower zoom lengths, example 24mm equiv for milky way and 200 mm for nebulae. If I power off, I have to redo the focussing procedure. Am I missing anything?