Simply questioning: if I choose the smallest aperture and focus to infinity - it doesn't really matter about where the focal point is through the viewfinder, does it?
Someone ought to actually answer the question.
Yes, it could matter, if focused at infinity. The depth of field extends behind, and also in front of the focused distance. The distance beyond infinity is wasted, so focusing at infinity is not necessarily the theoretical best choice.
The concept you seek is called the hyperfocal distance. This is the distance, when focused there, the depth of field extends behind to infinity, and also extends in front closer too.
Hyperfocal distance places the focused distance within the depth of field, so depth extends both ways. The value depends on focal length of the lens.
For example (making up ballpark numbers), maybe at f/16, you focus at 40 feet. The depth of field might then extend from about 20 feet to infinity.
It is not necessarily a huge difference, more subtle. If focused at infinity, the depth of field might extend from about 30 feet, but 20 is less than 30.
Hyperfocal distance - Wikipedia, the free encyclopedia