Mega Review of the best 8x42... (1 Viewer)

Alexis Powell · Jul 6, 2011

Steve C said:
...Experienced users of alpha glass, in my experience, ALWAYS have some preconcieved derogatory notion of the quality of this or that particular Chinese "clone".

Oh, surely not, and not me...not me. I think there is an equally large contingent among experienced users who are rooting for the underdog, and who take delight in instruments without sex appeal that just perform well.

--AP

ivan86 · Jul 6, 2011

elkcub said:
If you're referring to my hypothetical example, I don't know what you mean by "better overall scores," since the hypothetical SV ranked 3, 3, and 2, but came out first on an aggregate basis. Make sense to you?

Im sorry, i think is better for me write in italian and then you use google translate 3

For me, it make sense because even if SV ranked 3, 3 and 2, (in your example) is the binocular that got more points on average.
Maybe is strange, but "i dont care" about what the single partecipant think.
Any partecipant could have his personal opinion on the best bino on market BUT if the SV on avarange has been rated better, then for me SV is better even if noone of the single partecipant rated it 1°.

In this kind of review WIN the bino that got on avarange better scores.

This method ofc is less influenced by single opinions, thats why i think that it can bypass most of the personal preconceptions or psychological effects.

Maybe im thinking bad but whats your idea ? any link with methods?

elkcub said:
Again, no offense intended.

Dont worry, i agree with you on a lot of things, i do not want to defend my review, im just trying to explain my point of view.

If i have understood, you prefer a method that show better the single opinions of the partecipants.

Maybe i can make a gaussian distribution of the scores, with in the middle the average frequency, in short a graphical representation of the standard deviation.

I work and study, so i will need some time.

greets

NWBirder · Jul 6, 2011

ivan86 said:
Maybe is strange, but "i dont care" about what the single partecipant think.

:t: It will be so boring if everyone on this forum voices homogeneously the same opinion.

Kammerdiner · Jul 6, 2011

Alexis Powell said:
Oh, surely not, and not me...not me. I think there is an equally large contingent among experienced users who are rooting for the underdog, and who take delight in instruments without sex appeal that just perform well.

--AP

Agreed. In fact, if I look at a shirt with a polo pony embroidered on it you can rest assured I ain't buying it. I can find better at the thrift store. I don't pay squat for labels.

And I sure as heck didn't buy a Swaro SV because it had a cast pewter (or whatever it is) birdie stuck on it. Couldn't care less. Mind you the quality is, overall, just amazing, and that includes, up to a point, cosmetics. Makes my Zen ED2 look like what it is. Rushed and a bit sloppy.

And yet, speaking of clones, take a look at the discussion of reverse engineered Zen eyepieces over on the spotting scope forum. Makes you wonder if you really want to buy "fenced" goods. There is excellence, and then there is stealing excellence. I'm not much interested in buying stolen goods myself, but you can save some $$ that way.

As for the testing: remember that elkcub had to absolutely reverse two reviewers rankings to get the desired results. Not entirely likely with experienced reviewers. He does, however, make a legitimate point about a worst case scenario.

Mark

typo · Jul 6, 2011

ivan86 said:
Maybe i can make a gaussian distribution of the scores, with in the middle the average frequency, in short a graphical representation of the standard deviation.

Ivan please do not do that. I'm sorry we have failed to make ourselves clear. Perhaps if you read about "metodi parametrici" and "metodi non parametrici " (if Google translate works this time) you might understand what we are suggesting more clearly.

David

ivan86 · Jul 6, 2011

typo said:
Ivan please do not do that. I'm sorry we have failed to make ourselves clear. Perhaps if you read about "metodi parametrici" and "metodi non parametrici " (if Google translate works this time) you might understand what we are suggesting more clearly.

it worked :t: maybe i got your point.
I have not read this:

typo said:
Basically, you can't get more out than you put in, and if nine independent expert evaluations were put in, then, by golly, nine expert conclusions must come out.

Correct me if im wrong:
- i used a non-parametric method, because i gave to partecipants a table for "a poll". But then i used a parametric method to get the outcome.
This is a mistake because i cannot use a parametric method to get the outcome when i alredy used a non-parametric method to get the scores.

right? this is your point ?

if so, you are absolutely right but i needed to synthesize the review, and i could not find any other method decent (not good but decent...).

greets

typo · Jul 6, 2011

Kammerdiner said:
As for the testing: remember that elkcub had to absolutely reverse two reviewers rankings to get the desired results. Not entirely likely with experienced reviewers. He does, however, make a legitimate point about a worst case scenario.

Mark,

I think Ed was simply illustrating that scoring and totalling is a far from ideal way to report such a study. It hides the spectrum of opinion and potentially distorts the results. I'm not smart enough to get into the statistical argument why ranking is the better method, but like Ed I would like to see the results table for each tester. We all know how diverse opinion can be on this forum. It's a shame to condense it down to a homogeneous view which happens in virtually all mega-tests. As it is, I see some anomalies that I would like to understand better. Is this due to divided opinion or the consensus view. Do some, if any, share my relative views on the pairs I've tried. They may serve as a benchmark for others I might wish to look at. There's plenty of reason to hope for information.

Ivan and the others have a very detailed record of the study. I would encourage them to share it.

David

typo · Jul 6, 2011

ivan86 said:
Correct me if im wrong:
- i used a non-parametric method, because i gave to partecipants a table for "a poll". But then i used a parametric method to get the outcome.
This is a mistake because i cannot use a parametric method to get the outcome when i alredy used a non-parametric method to get the scores.

right? this is your point ?

Yes! Well done.

Ed was suggesting we would like " non parametrici" results for each participant as well.

Many thanks,

David

Pileatus · Jul 6, 2011

The best "result" would be to ask each reviewer to rank the bins in the order they would purchase them, regardless of cost.

denco@comcast.n · Jul 7, 2011

Pileatus said:
The best "result" would be to ask each reviewer to rank the bins in the order they would purchase them, regardless of cost.

Exactly. Just like they do in alot of automobile reviews.

ronh · Jul 7, 2011

The mega review was huge fun to read, and certainly tells something. But numberizing soft opinions just doesn't work very well. I'd like it if each reviewer wrote a short piece summarizing his findings, like two or three of the guys did. They could say what they found outstanding, without getting into the ranking of things that are nearly the same, or merely average.
Ron

denco@comcast.n · Jul 7, 2011

ronh said:
The mega review was huge fun to read, and certainly tells something. But numberizing soft opinions just doesn't work very well. I'd like it if each reviewer wrote a short piece summarizing his findings, like two or three of the guys did. They could say what they found outstanding, without getting into the ranking of things that are nearly the same, or merely average.
Ron

That's an excellent idea.

ivan86 · Jul 7, 2011

Ok good, thank you typo.

Give us some time, we will share at least some table.

greets

elkcub · Jul 7, 2011

Ivan,

I'm very pleased you're still responding. After giving thought to your comments on post #122, I'd like to suggest the following non-parametric approach. It involves a little bit of study and rethinking on your part, but it might be worth considering.

As mentioned before, you collected non-parametric data which allows tabulation of the several experts' preferences for rank ordering the sample binoculars. If you switch your focus of interest from the binoculars themselves to the expert's opinions, a natural question that arises is the degree to which the experts agree or disagree. I happen to like how you used Likert scales, but the very same question of ranking agreement would arise even if Pileatus and Denco@ obtained direct ratings from K individuals. (The two approaches don't produce the same thing, incidentally, but if you can still ask your experts to make direct rankings they could be used for validation purposes.)

The best non-parametric method to ascertain the over-all agreement among K sets of rankings of N objects is known as the Kendall Coefficient of Concordance: W. A brief summary can be found here: http://en.wikipedia.org/wiki/Kendall's_W, but a more informative discussion can be found in many stat books. I would recommend Siegal's "Non-parametric Statistics." It's really easy to read and understand.

If effect, the W statistic summarizes the ranking table. If all raters agree, W = 1. If they completely disagree W => 0.

Here is what Siegel says about the interpretation of W: "A high or significant value of W may be interpreted as meaning that the observers or judges are applying essentially the same standard in ranking the N objects under study. Often their pooled ordering may [then] serve as a "standard," especially when there is no relevant external criterion for ordering the objects."

I hope this helps. You have some good data to work with, — but, only a rare person would even consider taking someone else's advise under such conditions.

Thanks,
Ed

ivan86 · Jul 8, 2011

Thank you Ed, if is not a problem, in some time i will ask you some more explanation. At this time, i must first learn a few new things, then i will contact you.

greets

elkcub · Jul 8, 2011

Assolutamente, il mio amico, in qualsiasi momento.

Grazie,
Ed

Mega Review of the best 8x42... (1 Viewer)

More options

Alexis Powell

Natural history enthusiast

ivan86

Active member

NWBirder

Just Need One More Pair

Kammerdiner

Well-known member

typo

Well-known member

ivan86

Active member

typo

Well-known member

typo

Well-known member

Pileatus

"Experientia Docet”

[email protected]

Well-known member

ronh

Well-known member

[email protected]

Well-known member

ivan86

Active member

elkcub

Silicon Valley, California

ivan86

Active member

elkcub

Silicon Valley, California

Similar threads

Users who are viewing this thread