From ShotStat
Revision as of 13:33, 25 May 2015 by David (talk | contribs)
Jump to: navigation, search

Herb, 4/19/2015

RE: "Extreme Spread is not only a statistically inefficient measure but also one frequently and easily abused."

The most frequent abuse of extreme spread is chasing the "best group size" (the smallest group). The smallest group size is absolutely meaningless. The valid estimator is the average group size. If you want a smaller group size, just shoot more groups. Sooner or later you'll get lucky and shoot yet an even smaller group by pure luck.

Herb 5/11/2015

Danielson's 2-shot method is very inefficient. Assuming that horizontal and vertical deflection are both Gaussian and equal, and that the correlation coefficient is zero, then the gold standard is the radial standard deviation which is 100% efficient. In Danielson's 2-shot method he analyzed two different brands of ammo. He used 24 shots of each type, but only got 12 measurements per type. Combining all 24 shots for each type and analyzing using the Rayleigh model would be 100% efficient.

I believe the example worked out in this spreadsheet shows how 2-shot samples can be transformed to provide an efficient sample set for the Rayleigh model. The only trick to note is that each pair represents two observations, not just one. Thus from 24 shots we have 24 radii measurements (though only 12 are unique) and this allows us to compute the Rayleigh MLE for a 24-shot sample. If there is an error in that math or example please note it. David (talk) 13:00, 15 May 2015 (EDT)
The problem is that for 24 shots the Rayleigh method would have one average position for the horizontal and vertical position. By just cutting the difference between pairs of shots in half, you have 12 average positions for the horizontal and vertical deflection. User: Herb 15 May 2015, 6:24 EST

The "best" number of shots per group depends on the % of flyers. No flyers, 5-7 shots are about the same and are "best". A high % of flyers would mean that lower number of shots per group would be better.

Can you describe a statistically unbiased method of identifying flyers? David (talk) 13:00, 15 May 2015 (EDT)
You can't describe a distribution for flyers since the distribution function for flyers or any of its parameters are unknown. All that can be done is to "trim" the data based on the analytical model being used. So essentially you'd set a clip level and throw out the "worse" (largest) 5% of the measurements based on simulation. Probably multiple ways to trim data too. For instance think of group size for 5-shot groups. You could set clip levels at the highest and lowest 2.5% levels based on simulation. You could also compare 5-shot group size to 4-shot group size for every group. So let's assume that an "average" 5-shot group is 1 inch. I have a 5-shot group with is 6 inches, but has 4 shots that are in a group of 0.75 inches. The 5-shot to 4-shot ration would put the is group well beyond the clip level of the largest 5% of simulated groups and thus the 5-shot group could be viewed as "abnormal". That is the rub with statistics. I can calculate the exact probability of flipping a penny and getting a hundreds tails in a row even though such a result is practically impossible. So if someone did flip a hundred tails in a row, you would have to have to be very skeptical that the flips were fair. You can't use statistics to "prove" that the flips were unfair, you can only infer that the result was highly unusual at some confidence level. User: Herb 15 May 2015, 6:24 EST

Think of Danielson's method like an ANOVA problem. The ((difference in distance)/2)^2 between two shots in a group gives variance within a group. But each group of the 12 groups has it own center. There is also an "overall center" which is the center of the 12 centers. Thus there is also a variance between centers for the 12 groups. For just two shots per group the variance between group centers is very significant compared to the variance within a group.

It is an egregious error to think of Danielson's method as "efficient". Yes two-shot group size for just one group is 100% efficient. The inefficiency is the result of having multiple two-shot groups. 100% efficiency would be to use the 24 shots in a single Rayleigh distribution analysis which would have one center for all 24 shots and 24 radii from that lone center.

The situation where group size using 2-shot groups would be most appropriate would be when testing for a high percentage of "outliers". Here I'm using "outlier" to mean a real aberration, not just a shot from the distribution with a wide spread. Otherwise 5-7 shots per group would be more statistically efficient.

Herb (talk) 15:59, 24 May 2015 (EDT)

When I did the math I found that, despite the fact that breaking the sample into groups of two shots throws away information about the combined center, the Rayleigh estimate and confidence intervals are identical. I believe this is easy enough to show with the Rayleigh MLE formula. I'll put this on my queue to prove. David (talk) 18:05, 24 May 2015 (EDT)
No way that is true. I'm at a loss as to how to try to explain the situation better.
Herb (talk) 21:32, 24 May 2015 (EDT)
You should smell a rat in all of this. Look at page 4 of pdf file for figure "Estimation of Efficiency By Group Size" (lower left of page 44 of text). This assumes no flyers. At first as there are more shots in a group the efficiency increases because the group center becomes a better estimate of the population center. However since the group size "only uses" two shots, as the group gets more shots the efficiency goes down. ("Only uses" isn't absolutely true but overall trend is absolutely correct.) The efficiency of 5-7 shot groups is about same and "most efficient" (again neglecting flyers which is a huge assumption).
Herb (talk) 22:59, 24 May 2015 (EDT)
I had a chance to review this and you're right: I was making too strong a claim. What I had found is an isomorphism between the T-test and the Rayleigh confidence measure for hypothesis testing. So no: you can't get an efficient estimate of precision using 2-shot groups. And, obviously even for hypothesis testing you will be able to reject hypotheses with greater certainty if you sample from larger groups. I'll remove that section from the main page! David (talk) 13:33, 25 May 2015 (EDT)