Hmmm, an interesting thread has developed while I have been away on my penultimate summer vacation week...
Jean-Charles: You have done good work and obviously also learned a lot doing it. Your conclusions are, as far as I can judge, very sound and sane. The first thing I would like to add pertains to relating boosted results to subjective viewing experience. Namely, it makes a lot of difference which of your eyes is the dominant one. If the binocular has lots of aberrations in the barrel which serves your dominant eye, you will tend to find it poor, but if the the dominant eye receives a clean image, you can stand a fair amount of aberrations in the non-dominant eye's barrel before you begin to be seriously bothered by a perception of a poor overall image.
The second thing is that sample variation does not render meaningless the differences in transmission efficiency, stray light control, ergonomics, field of view etc., which are more design-specific than unit-specific, although I do tend to agree that an (almost) aberration-free binocular in general is more pleasant to view with (and definitely sharper) than an otherwise better and more modern but less perfect one.
Thirdly, although I have not specifically concentrated on chromatic aberration with respect to individual unit variation in binoculars, I have noticed that in telescopes there is visible (to the naked eye) difference in CA between units in the sense that some units tend to show uncharacteristically large amounts of CA. I would suspect that the same goes for binoculars, but the differences might be hard to detect.
Elk: I wish it were as simple as testing 3 (or five or any other smallish finite number) specimen to get a fair picture of sample variation within a given brand and model. Obviously, three is better than one, but how do we now that variation stays constant over time or what factors contribute to it? Is it mostly a matter of which worker worked the shift when the unit was assembled, who was doing the final product quality check, which batch of glass was used for that unit's objective lenses or what? All sorts of interesting test protocols could be devised, many of them yielding more meaningfull results than what are presently offered, but they would all fall apart at the point when the earnest reviewer would approach the manufacturer or the importer with a request along the lines of: "could we please have ten units of your XXX binocular on loan for a comparative test where we try to see how the average quality of your product compares with the average quality of brand YYY" Of course, if you could get Consumer Reports on board and have the wherewithall to actually PURCHASE randomly a large enough number of everything you wanted to test, the problem would be solved. This way you could also make sure that the manufacturers do not send you tuned-up or pre-selected cherries.
Walter: A good to excellent specimen of an older model Leica or Zeiss will most definitely give a sharper image than a mediocre to poor specimen of a most recent model sibling. Same goes for contrast. Brightness and color balance as well as freedom from flare are much more reliably design-specific, and I would be very surprised to hear that someone would find a Trinovid brighter than an Ultravid, for example.
Kimmo
