The middle apertures are often the very-sharpest, best-performing apertures on many lenses, and on many cameras. These days, as we get into higher and higher megapixel count sensors, the smaller f/stops, like f/5.6 and f/8 are already showing diffraction, which is an optical phenomenon that causes a loss of sharpness. So...with today's 24-MP sensors, many good lenses produce their sharpest images at surprisingly wide apertures, like say, f/4 to f/5.6 or thereabouts.
Another issue is sensor dust spots: at the moderate apertures, sensor dust spots appear on-screen as about 3mm, round, soft, gentle-edged, faint blobs--which are VERY easy to clone out using the clone stamp tool, and on areas with a lot of detail, these sensor dust blobs can often just be left in the shot, since they are very hard to spot unless they appear on broad, smooth areas, like the sky for example. At f/11, the same sensor dust blob will generate a DARK, dense, "speck", which is very noticeable, no matter where in the frame it happens to fall.
Another benefit of the middle apertures, like f/4 to f/6.3, is that they tend to keep the shutter speeds HIGHER than if a smaller aperture were used. Again, this is an issue the more that we move into the high-resolution era of 16 and 20 to 24, to even 36 MP captures. With a 24MP camera and a 70-300mm lens at 300mm length, at 1/320 second and f/5.6, if you look CLOSELY, you will see that people who are just moving normally are, fairly often, actually NOT 100% sharp on moving extremities.
It's subtle, but the difference between say 1/125 second and 1/500 second can make a big difference in perceived sharpness on many scenes, now that wee have moved away from the 6 to 8 megapixel era...