Difference between revisions of "Extreme Spread"

From ShotStat
Jump to: navigation, search
(Mode, Median, Mean, Variance, %RSD of Mean)
(Experimental Summary)
(31 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
{|align=right
 +
  |__TOC__
 +
  |}
 
= Experimental Summary =
 
= Experimental Summary =
  
Line 13: Line 16:
 
| Assumptions
 
| Assumptions
 
|
 
|
* Ideally the shots would follow a Rayleigh Distribution  
+
* Ideally the shots would have a circular pattern (i.e. the shots would have the Rayleigh Distribution).
** <math>\bar{h} \sim \mathcal{N}(\bar{h},\sigma_h^2), \bar{v} \sim \mathcal{N}(\bar{v},\sigma_v^2)</math>
+
** <math>h \sim \mathcal{N}(\bar{h},\sigma_h^2), v \sim \mathcal{N}(\bar{v},\sigma_v^2)</math>
 
** Horizontal and vertical dispersion are independent.  
 
** Horizontal and vertical dispersion are independent.  
 
** <math>\sigma_h = \sigma_v</math> (realistically <math>\sigma_h \approx \sigma_v</math>)
 
** <math>\sigma_h = \sigma_v</math> (realistically <math>\sigma_h \approx \sigma_v</math>)
 
** <math>\rho = 0</math>
 
** <math>\rho = 0</math>
** <math>PDF_{r_i}(r) = \frac{r}{\sigma^2}e^{-r^2/2\sigma^2}</math>
+
** <math>PDF_{r_i}(r) = \frac{r}{\Re^2}e^{-r^2/2\Re^2}</math>
:: '''Note:''' It is not necessary to fit <math>\sigma</math> to calculate the Figure of Merit.
+
:: '''Note:''' It is not necessary to calculate the COI, nor the constant <math>\Re</math>, to calculate the Extreme Spread.
 
* No Fliers
 
* No Fliers
 
|-
 
|-
 
| Data transformation
 
| Data transformation
| Identify two holes, <math>i, j</math> which are the farthest apart.  
+
| Identify two holes, <math>i, j</math> which are the farthest apart and measure <math>ES</math>.
 +
&nbsp;<math>ES = \sqrt{(x_i - x_j)^2 + (y_i - y_j)^2}</math>
 
|-
 
|-
 
| Experimental Measure
 
| Experimental Measure
| <math>ES = \sqrt{(x_i - x_j)^2 - (y_i - y_j)^2)}</math>,
+
| <math>ES</math>
 
|}
 
|}
  
Line 35: Line 39:
 
== Assumptions ==
 
== Assumptions ==
  
None are needed to make measurement. However making assumptions about the dispersion will enable theoretical predictions about the measurement.
+
None are needed to make measurement. However some points are worth considering.
 +
 
 +
* The same ES measurement could result from a vertical group to a round group. If the shooting process can vary that much then the ES measurement won't give any indication of the change.
 +
 
 +
:: If the shot patterns aren't "fairly" round, then using the measurement makes little sense. For instance if muzzle velocity variations are severe, then the vertical range will dominate the ES measurement. Muzzle velocity variations would correlate better with vertical range than with ES.
 +
 
 +
* Making assumptions about the dispersion will enable theoretical predictions about the ES measurement. It must be realized that the theoretical solution, assuming the Rayleigh distribution and using Monte Carlo simulation, isn't some arbitrary goal, it is the best case scenario.
  
 
== Data transformation ==
 
== Data transformation ==
  
The data transformation for a human has simple requirements, just the ability to locate the holes which are the furthest apart. If the target has a ragged hole it can be a bit tricky, but the edges of the hole should have enough curvature to make shot location possible.
+
The data transformation for a human has simple requirements, just the ability to locate the holes which are the furthest apart and measure the distance between them. If the target has a ragged hole it can be a bit tricky, but the edges of the hole should have enough curvature to make shot location possible.
  
== Experimental Measure ==
+
If measuring on the range, then the center of the hole is difficult to locate. Typically a vernier caliper (cheap is fine!) would be used to measure the distance from the outside edges of the holes, then the bullet caliber subtracted to get a c-t-c measurement.
  
If measuring on the range, then the center of the hole is difficult to locate. Typically a vernier caliper would be used to measure the distance from the outside edges of the holes, then the bullet caliber subtracted to get a c-t-c measurement.  
+
:{| class="wikitable"
 +
| [[File:Bullseye.jpg|50px]] A cheap ($10-$20) vernier caliper works fine. There is no need for a $2,000 one that measures to 1/10,000<sup>th</sup> of an inch and has National Bureau of Standards calibration. The vernier caliper is nice for the c-t-c measurement because the knife edges will be parallel and won't obscure the edges of the bullet hole. Thus it is easy to accurately place both of the knife edges on a tangent to the curved bullet holes.  
 +
|}
  
 
If using a computer then the center location would be a matter programming. For example a mouse might be used simply to point out the holes, or to drop a dot at the center of the hole, or to drag a circle over the hole. The computer would then make the c-t-c measurement.
 
If using a computer then the center location would be a matter programming. For example a mouse might be used simply to point out the holes, or to drop a dot at the center of the hole, or to drag a circle over the hole. The computer would then make the c-t-c measurement.
 +
 +
== Experimental Measure ==
 +
 +
No calculation needs to be done to get the measurement. The single physical measurement is the data sought for the target.
  
 
== Outlier Tests ==
 
== Outlier Tests ==
  
= Theoretical <math>ES</math> Distribution =
+
= Theoretical Evaluations =
 +
 
 +
== Dispersion Follows Rayleigh Distribution ==
  
Assuming that the shots are Rayleigh distributed allows us to make some theoretical estimates. Since the distribution is positively skewed: Mean > Median > Mode.  
+
Assuming that the shots are Rayleigh distributed allows us to make some theoretical estimates.  
  
 
{| class="wikitable"  
 
{| class="wikitable"  
|+ Theoretical <math>ES</math> Distribution
+
|+ Theoretical <math>ES</math> Distribution of <math>n</math> shots
 
|-
 
|-
 
| Parameters Needed
 
| Parameters Needed
 
|  
 
|  
 
|-
 
|-
| <math>PDF(r; \sigma)</math>
+
| <math>PDF_{ES}(r; n)</math>
| no direct evaluation, must be simulated via Monte Carlo
+
| no deterministic solution, must be simulated via Monte Carlo
 
|-
 
|-
| <math>CDF(r; \sigma)</math>
+
| <math>CDF_{ES}(r; n)</math>
| no direct evaluation, must be simulated via Monte Carlo
+
| no deterministic solution, must be simulated via Monte Carlo
 
|-
 
|-
| Mode of PDF)
+
| Mode of <math>PDF_{ES}</math>
| depends on <math>n</math>, in general Mode increases as number of shots increases.  
+
| depends on <math>n</math>, Mode increases as number of shots increases.  
 
|-
 
|-
| Median of PDF
+
| Median of <math>PDF_{ES}</math>
| depends on <math>n</math>, in general Median increases as number of shots increases.
+
| depends on <math>n</math>, Median increases as number of shots increases.
 
|-
 
|-
| Mean of PDF
+
| Mean of <math>PDF_{ES}</math>
| depends on <math>n</math>, in general Median increases as number of shots increases
+
| depends on <math>n</math>, Median increases as number of shots increases
 
|-
 
|-
 
| Variance
 
| Variance
| no direct evaluation, must be simulated via Monte Carlo
+
| no deterministic solution, must be simulated via Monte Carlo
 
|-
 
|-
| Variance Distribution
+
| Efficiency
|
+
| depends on <math>n</math>, best about 5-7 shots
 
|-
 
|-
 
| (h,v) for all points?
 
| (h,v) for all points?
Line 89: Line 107:
  
  
== Parameters Needed ==
+
=== Parameters Needed ===
  
== PDF ==
+
=== PDF ===
  
== CDF ==
+
=== CDF ===
  
== Mode, Median, Mean, Standard Deviation, %RSD ==
+
=== Mode, Median, Mean, Standard Deviation, %Rel Std Dev ===
  
{| class="wikitable"
+
Since the distribution is positively skewed:
|+ Table columns for "ES Values from Monte Carlo Simulation" Table
+
<blockquote>Mean > Median > Mode</blockquote>
|-
+
 
| number of shots
+
"Normality Error"
| the number of shots used per target
+
As sort of a crude indication of normality let's use the value:
|-
 
| Mode
 
| The peak of the distribution
 
|-
 
| Median
 
| The 50<sup>th</sup> percentile
 
|-
 
| Mean
 
| the average value over the whole distrubtion
 
|-
 
| "Normality Error"
 
| As sort of a crude indication of normality let's use the value:
 
  
 
"Normality Error" = <math> \frac{\frac{CDF(5) - CDF(95)}{2} - Mean}{Mean} {\cdot 100}</math>
 
"Normality Error" = <math> \frac{\frac{CDF(5) - CDF(95)}{2} - Mean}{Mean} {\cdot 100}</math>
Line 120: Line 126:
 
* + value means positively skewed,  
 
* + value means positively skewed,  
 
* - value means negatively skewed.  
 
* - value means negatively skewed.  
|-
 
| Standard
 
Deviation
 
|
 
|-
 
| %RSD
 
| The relative standard deviation which is a ratio of the standard deviation to the mean as a percentage. 
 
|}
 
  
 
The point of the "Normality Error" is to give the reader a quinsy feeling about using Student's T-Test for groups with few shots, or the average of a small number of targets.  
 
The point of the "Normality Error" is to give the reader a quinsy feeling about using Student's T-Test for groups with few shots, or the average of a small number of targets.  
Line 142: Line 140:
 
! Standard
 
! Standard
 
Deviation
 
Deviation
! %RSD
+
! %Rel Std Dev
 
|-
 
|-
 
| 2
 
| 2
Line 233: Line 231:
 
This is another example to warn the reader. Just because you can calculate a standard deviation doesn't mean that a Student's T Test will work. A typical 95% confidence Interval for an individual ES measurement is <math>\pm 1.96 \sigma</math> and for a 2-shot group that is:
 
This is another example to warn the reader. Just because you can calculate a standard deviation doesn't mean that a Student's T Test will work. A typical 95% confidence Interval for an individual ES measurement is <math>\pm 1.96 \sigma</math> and for a 2-shot group that is:
 
:&nbsp;<math>\pm 1.96 \cdot 52.6\% = \pm 103.1\%</math> of the measurement
 
:&nbsp;<math>\pm 1.96 \cdot 52.6\% = \pm 103.1\%</math> of the measurement
so the lower confidence interval would be at '''-3.1% !!!''' The nonsensical result is because the distribution is skewed. A negative extreme spread measurement is impossible (except perhaps in Australia which is upside down from the USA...)
+
so the lower confidence interval would be at '''-3.1% !!!''' The nonsensical result is because the distribution is skewed. A negative extreme spread measurement is impossible. It isn't the standard deviation that is wrong, it is the assumption that the confidence interval would be <math>\pm 1.96 \sigma</math> that is the problem. Since the distribution is skewed, the low side of the confidence interval at the 2.5 percentile is at ?? and the high side of the confidence interval at the 97.5 percentile is at +??.  
  
 
At 5 shots the T-test is reasonable, and at 10 shots pretty good.  
 
At 5 shots the T-test is reasonable, and at 10 shots pretty good.  
Line 240: Line 238:
  
 
* Estimate ES values for different group sizes.
 
* Estimate ES values for different group sizes.
: Given Mean of 5 shots = 3.066 and measured ES is 1.53 inches
+
:: A 3-shot group would be given by measured size times ratios of the Means from the table
:: A 3-shot group would be given by measured size times ratios of Means
 
 
:::&nbsp;<math>1.53 \frac{2.406}{3.066} = 1.20</math> inches
 
:::&nbsp;<math>1.53 \frac{2.406}{3.066} = 1.20</math> inches
:: A 10-shot group would be given by measured size times ratios of Means
+
:: A 10-shot group would be given by measured size times ratios of Means from the table
 
:::&nbsp;<math>1.53 \frac{3.813}{3.066} = 1.90</math> inches
 
:::&nbsp;<math>1.53 \frac{3.813}{3.066} = 1.90</math> inches
  
* Estimate expected standard deviation from the measured ES value
+
* Estimate the expected standard deviation from the measured ES value
: Given Mean of 5 shots = 3.066 and measured group size is 1.53 inches
+
:: The %RSD value for 5-shots is 27.0% so:
:: The %RSD value for 5-shots is 0.270 so:
+
:::&nbsp;<math>\hat{s} = 1.53 \text{ inches} \cdot 0.270 = 0.413 \text{ inches} </math>
:::&nbsp;<math>\hat{s} = 1.53 \cdot 0.270 = 0.413 </math>
 
  
 
* Estimate the expected Standard Deviation of the average of 4 targets
 
* Estimate the expected Standard Deviation of the average of 4 targets
:&nbsp;<math>\text{SD}_{ES\ 4 \  Targets}\ = \frac{30.7\%}{\sqrt{4}} = 15.4\% </math>
+
:&nbsp;<math>\text{SD}_{ES\ 4 \  Targets}\ = \frac{27.0\%}{\sqrt{4}} = 13.5\% </math>
 +
 
 +
=== Variance ===
 +
 
 +
=== Efficiency ===
 +
[[File:Extreme Spread Relative Efficiency.png|300px|thumb|right|]]
  
== Sample Variance and Its distribution ==
+
The efficiency graph on the right is based on the number of groups, hence total shots, to get a 10% confidence interval.
 +
The efficiency depends on <math>n</math>, but it is best about 5-7 shots. Essentially there are two competing factors. First as the number of shots increases then the midpoint of the line segment which defines the MR is, on average, closer to the COI which improves efficiency. Second as the number of shots increases then it is increasingly unlikely that the next shot will increase the MR which decreases efficiency. The product of these two factors thus peaks at about 5-7 shots.
  
== Outlier Tests ==
+
Notice too that the figure utilizes fractional groups to define a smoother curve. In reality, especially for small samples, using all the shots would be important. Thus for ammunition which is packaged as 20 cartridges per box, then using 8 shot groups leaves 4 cartridges unused which is 20% "inefficient." 
 +
 
 +
This result is also assuming no fliers. If 5-7 shots is likely to give groups with multiple fliers, then less shots per group might be better.
 +
 
 +
So what is the optimal number of shots per group? '''''It depends...'''''
 +
 
 +
=== Outlier Tests ===
  
 
= See Also =
 
= See Also =
  
[[Dispersion Assumptions]] - A discussion of the different cases for shot dispersion
+
[[Projectile Dispersion Classifications]] - A discussion of the different cases for projectile dispersion
  
 
Other measurements practical for range use are:  
 
Other measurements practical for range use are:  

Revision as of 00:34, 17 June 2015

Experimental Summary

Given
  • set of n shots {\( (h_1, v_1), (h_2, v_2), ..., (h_n, v_n) \)}

All of the (h,v) positions do not need to be known so a ragged hole will suffice.

Assumptions
  • Ideally the shots would have a circular pattern (i.e. the shots would have the Rayleigh Distribution).
    • \(h \sim \mathcal{N}(\bar{h},\sigma_h^2), v \sim \mathcal{N}(\bar{v},\sigma_v^2)\)
    • Horizontal and vertical dispersion are independent.
    • \(\sigma_h = \sigma_v\) (realistically \(\sigma_h \approx \sigma_v\))
    • \(\rho = 0\)
    • \(PDF_{r_i}(r) = \frac{r}{\Re^2}e^{-r^2/2\Re^2}\)
Note: It is not necessary to calculate the COI, nor the constant \(\Re\), to calculate the Extreme Spread.
  • No Fliers
Data transformation Identify two holes, \(i, j\) which are the farthest apart and measure \(ES\).

 \(ES = \sqrt{(x_i - x_j)^2 + (y_i - y_j)^2}\)

Experimental Measure \(ES\)

Given

The requirements for this test are very basic. Just a target with \(n\) shots, and some measuring device. Assuming an Extreme spread of under 6 inches then a vernier caliper is used. A measurement is possible to a few thousandths of an inch which is vastly more precision than is usually required. From longer distance a ruler, or perhaps a tape measure.

Assumptions

None are needed to make measurement. However some points are worth considering.

  • The same ES measurement could result from a vertical group to a round group. If the shooting process can vary that much then the ES measurement won't give any indication of the change.
If the shot patterns aren't "fairly" round, then using the measurement makes little sense. For instance if muzzle velocity variations are severe, then the vertical range will dominate the ES measurement. Muzzle velocity variations would correlate better with vertical range than with ES.
  • Making assumptions about the dispersion will enable theoretical predictions about the ES measurement. It must be realized that the theoretical solution, assuming the Rayleigh distribution and using Monte Carlo simulation, isn't some arbitrary goal, it is the best case scenario.

Data transformation

The data transformation for a human has simple requirements, just the ability to locate the holes which are the furthest apart and measure the distance between them. If the target has a ragged hole it can be a bit tricky, but the edges of the hole should have enough curvature to make shot location possible.

If measuring on the range, then the center of the hole is difficult to locate. Typically a vernier caliper (cheap is fine!) would be used to measure the distance from the outside edges of the holes, then the bullet caliber subtracted to get a c-t-c measurement.

Bullseye.jpg A cheap ($10-$20) vernier caliper works fine. There is no need for a $2,000 one that measures to 1/10,000th of an inch and has National Bureau of Standards calibration. The vernier caliper is nice for the c-t-c measurement because the knife edges will be parallel and won't obscure the edges of the bullet hole. Thus it is easy to accurately place both of the knife edges on a tangent to the curved bullet holes.

If using a computer then the center location would be a matter programming. For example a mouse might be used simply to point out the holes, or to drop a dot at the center of the hole, or to drag a circle over the hole. The computer would then make the c-t-c measurement.

Experimental Measure

No calculation needs to be done to get the measurement. The single physical measurement is the data sought for the target.

Outlier Tests

Theoretical Evaluations

Dispersion Follows Rayleigh Distribution

Assuming that the shots are Rayleigh distributed allows us to make some theoretical estimates.

Theoretical \(ES\) Distribution of \(n\) shots
Parameters Needed
\(PDF_{ES}(r; n)\) no deterministic solution, must be simulated via Monte Carlo
\(CDF_{ES}(r; n)\) no deterministic solution, must be simulated via Monte Carlo
Mode of \(PDF_{ES}\) depends on \(n\), Mode increases as number of shots increases.
Median of \(PDF_{ES}\) depends on \(n\), Median increases as number of shots increases.
Mean of \(PDF_{ES}\) depends on \(n\), Median increases as number of shots increases
Variance no deterministic solution, must be simulated via Monte Carlo
Efficiency depends on \(n\), best about 5-7 shots
(h,v) for all points? yes for simulation.
Symmetric about Mean? No, skewed to larger values.

More symmetric about mean as the number of shots increases.


Parameters Needed

PDF

CDF

Mode, Median, Mean, Standard Deviation, %Rel Std Dev

Since the distribution is positively skewed:

Mean > Median > Mode

"Normality Error"
As sort of a crude indication of normality let's use the value:

"Normality Error" = \( \frac{\frac{CDF(5) - CDF(95)}{2} - Mean}{Mean} {\cdot 100}\)

So we measure half the distance between the 5th percentile and the 95th percentile to determine where the Mean should be if the distribution was symmetrical, and determine the % error based on the actual value of the mean.

  • + value means positively skewed,
  • - value means negatively skewed.

The point of the "Normality Error" is to give the reader a quinsy feeling about using Student's T-Test for groups with few shots, or the average of a small number of targets.


Theoretical ES Values from Monte Carlo Simulation Distribution
number of shots Mode Median Mean "Normality Error" Standard

Deviation

%Rel Std Dev
2 1.772 0.932 52.6%
3 2.406 4.95% 0.887 36.9%
4 2.787 0.856 30.7%
5 3.066 3.06% 0.828 27.0%
6 3.277 0.806
7 3.443 0.783
9 3.710 0.754
10 3.813 0.745
20
30 4.788 1.63% 0.745 15.6%

The tabular values can be used in a number of ways:

Estimate a 95% confidence Interval for Given 2-shot groups based on one ES measuremnt

So a 2-shot group has been measured. If the measured value is accepted as the true value, what would the standard deviation of multiple 2-shot groups be?

This is another example to warn the reader. Just because you can calculate a standard deviation doesn't mean that a Student's T Test will work. A typical 95% confidence Interval for an individual ES measurement is \(\pm 1.96 \sigma\) and for a 2-shot group that is:

 \(\pm 1.96 \cdot 52.6\% = \pm 103.1\%\) of the measurement

so the lower confidence interval would be at -3.1% !!! The nonsensical result is because the distribution is skewed. A negative extreme spread measurement is impossible. It isn't the standard deviation that is wrong, it is the assumption that the confidence interval would be \(\pm 1.96 \sigma\) that is the problem. Since the distribution is skewed, the low side of the confidence interval at the 2.5 percentile is at ?? and the high side of the confidence interval at the 97.5 percentile is at +??.

At 5 shots the T-test is reasonable, and at 10 shots pretty good.

Given ES of one 5-shot group is 1.53 inches

  • Estimate ES values for different group sizes.
A 3-shot group would be given by measured size times ratios of the Means from the table
 \(1.53 \frac{2.406}{3.066} = 1.20\) inches
A 10-shot group would be given by measured size times ratios of Means from the table
 \(1.53 \frac{3.813}{3.066} = 1.90\) inches
  • Estimate the expected standard deviation from the measured ES value
The %RSD value for 5-shots is 27.0% so:
 \(\hat{s} = 1.53 \text{ inches} \cdot 0.270 = 0.413 \text{ inches} \)
  • Estimate the expected Standard Deviation of the average of 4 targets
 \(\text{SD}_{ES\ 4 \ Targets}\ = \frac{27.0\%}{\sqrt{4}} = 13.5\% \)

Variance

Efficiency

Extreme Spread Relative Efficiency.png

The efficiency graph on the right is based on the number of groups, hence total shots, to get a 10% confidence interval. The efficiency depends on \(n\), but it is best about 5-7 shots. Essentially there are two competing factors. First as the number of shots increases then the midpoint of the line segment which defines the MR is, on average, closer to the COI which improves efficiency. Second as the number of shots increases then it is increasingly unlikely that the next shot will increase the MR which decreases efficiency. The product of these two factors thus peaks at about 5-7 shots.

Notice too that the figure utilizes fractional groups to define a smoother curve. In reality, especially for small samples, using all the shots would be important. Thus for ammunition which is packaged as 20 cartridges per box, then using 8 shot groups leaves 4 cartridges unused which is 20% "inefficient."

This result is also assuming no fliers. If 5-7 shots is likely to give groups with multiple fliers, then less shots per group might be better.

So what is the optimal number of shots per group? It depends...

Outlier Tests

See Also

Projectile Dispersion Classifications - A discussion of the different cases for projectile dispersion

Other measurements practical for range use are:

  • Covering Circle Radius - about same precision as Extreme Spread if Rayleigh distributed
  • Diagonal - somewhat better precision than Extreme Spread if Rayleigh distributed
  • Figure of Merit - somewhat better precision than Extreme Spread if Rayleigh distributed