### On measurements accuracy

After thinking over a bit, it could be figured out that when cognizing and describing the world around us we, in the first place, are trying to understand the laws that rule the environment. Knowledge of a certain nature law and ability to correctly express it by a mathematical formula shall give us an opportunity to predict (calculate) various phenomena to a high precision in future.

We come to understand this fact pretty early. But we do not always realize that it is not enough. In these laws there are always certain constants and other, unknown beforehand values, which are necessary to be estimated (measured) experimentally. Let us take a simple example – the law of universal gravitation.

Its components, both interaction constant γ, and masses of bodies (M and m), do not come of nowhere, and they must be estimated experimentally. Just think of any other physical law you know. It definitely contains variables “introduced in there with our own hands”. I am saying this to remind you how it is important to understand how the accuracy of our measurements is estimated.

In our previous notes we were trying to explain that all the measurements have an error and how to estimate both “true” (the quotes are used because any final result always has an error) value and inaccuracy (error) of this estimation. You should have learned that both these tasks may be solved only by way of numerous independent measurements. At that, we have omitted the question on how the results of these independent measurements will behave (be distributed). Before we move on you have to understand what the measurement accuracy means and how it should be estimated. In other words you should comprehend and be able to answer the question – “I have measured the EAS intensity at the stations X and Y. They appeared to differ by only 4%. May I trust this data?” This question has been thoroughly considered in the theory of probability. It was shown that the distribution of the majority of independent measurements was described by the formula of normal or Gaussian distribution.

x - the result of the given measurement;

μ - “true” value of the measured variable (we take it as the average value received from all the measurements);

σ - the width or mean-square error of the given distribution.

I am not trying to encourage you to learn this formula by heart. Those who need it will do it in future. I have driven it here with the only purpose to convince you that there are formulas which may be used to predict the way our measurements errors shall be distributed and how the criteria of difference of two variables shall develop. I hardly need to say how important it is to sometimes understand whether the variables are truly differ or the effect has been caused by mere changes (fluctuations) due to poor accuracy of the measurements? My example which you have to work upon belongs to this particular area related to the question “Is the difference of 4% essential or not?”

In order to understand how to answer this question let us go back to the foregoing formula. It would be more convenient to us to depict it in the form of the following diagram (Fig. 1).

It doesn’t only show how this function looks, but it also shows which percent of the events gets into the interval from 1 to 5 variables of the mean-square error (σ), which we shall agree to call a measurement error. It is clear from the diagram that no matter how many events we would accumulate, but in interval:

- (± 1σ from the average value) will include only 68,3% of all measurements,
- (± 2σ) – (68,3+2*13,6 = 95,5%)
- (± 3σ) – (68,3+2*(13,6 +2,1) = 99,7%)
- (± 4σ) – now comes to (68,3+2*(13,6 +2,1+0,1) = 99,99%)

The above figures show us the following:

- Average variables (meaning our assessment of the “true” value of the unknown variable under measurement) of the statistically distributed variables may be considered to be different if they differ by more than 3 σ (3 errors), since the probability of a casual appearance of an event differing by the value exceeding 3σ is not high; it is as low as 0,3% (100-99,7)%.
- Nevertheless, the probability of appearance of the event being far different (beyond 5σ) from the average is low, BUT IT EXISTS! This is the reason why it is better not to make any conclusions by only one event of the statistic variable.

Now let us get back to our major issue: is the difference of 4% a real effect or not? The answer will be odd – we cannot say for sure: maybe “yes” and maybe “no” – all depends on what total number of events (selection) we shall use to make our conclusion. Let us suppose that there is really 4% more of EAS that falls to station № 2, which presumably may be located higher than station № 1. Let us consider three possible variants.

Variant №1 The data set lasted about 1 minute. The counting of station X (№1) was 48 and the counting of station Y (№2) was 52. We already know that the error of the average estimation is equal to √n or 6,8 for station №1 and 7,2 for station №2. A scientist (not a journalist) would put down the final results of this experiment in the following way:

The EAS counting rate determined by station №1 came to be 48 ± 6,8 counts, and the EAS counting rate determined by station №2 – 52 ± 7,2 counts, and this fact indicates that these values do not differ within the limits of these measurements (calculations) accuracy. The difference of the count is (52-48) = 4, and the possible error will be 14 (6,8+7,2) counts! What does this statement mean? It means that if we start to conduct a number of 1 minute measurements, we’ll receive a set of wide variety of values, the probability of which appearance will be described by the Gaussian distribution. Having accumulated much data from 1 minute sets, we shall receive normal (Gaussian) distributions around real, but still (unknown) to us values of 48/52 for each station. At that, these distributions will be wide and overlapping each other as it is shown in Fig.2 - red curve – the distribution of 1 minute counts at station №1, and blue curve is the same for station №2. What do these distributions mean? They show us within which values and with what probability one minute count data shall appear at these stations. Most often (in 68,3%) there will appear the event within the limits from ~44 to ~ 55 (± 1σ). More seldom, but still likely there may appear an event with the total amount of 30 counts in one minute, that is to say, all new measurements will “jump” inside this curve. At one time there will be the ratio of 45/55, the next selection will give a vice-versa ratio of 55/45 and so on and so forth – all these because of highly-overlapping curves, as it is well seen in Fig. 2.

Variant №2. Now we shall set data in every ~ 10 minutes. That would mean in average 480 and 520 counts at stations №1 and №2 provided the proportion of our example of 48% to 52% is maintained. The distribution of statistically possible variants, like in the previous case will be described by the Gaussian distribution but with different constants

(µ - average values will now be 480 and 520, and σ (mean-square error) – 22 and 23. And our intermediate results of measurement will look like as it is shown in (Fig.3). The data for station № 1 is 480 ± 22 and for station №2 – 520 ± 23, which again coincides within the limits of measurements (calculations) accuracy. And this becomes certain as soon as you understand that they may “jump” inside the respective Gaussian distribution. The difference of 40 is within the limits of 2σ = 45. All the explanations referred to the curves in the picture from the first variant remain in force. It is well seen that the curves are “drawn apart” more but they still visibly overlap, so it is hardly ever possible to say that 480 and 520 is a true difference according to the result of one or several data sets out of even 1000 counts!

Variant №3. Now we shall set data in every ~ 15 hours. During this time station №1 shall accumulate ~ 48000 ± 219 counts and station № 2 shall accumulate ~ 52000 ± 228 counts, and the difference of 52000/48000 in ~ 4000 counts shows a reliable (~ 9 errors) increase of the number of counts at the second station as compared with the first one. Look at Fig. 4, where there are the relevant Gaussian functions for the values of 48000 and 52000. The dispersion in the readings (if to count them in portions within the interval of 15 hours) shall persist, but it would be impossible to estimate the ratio in the wrong way! The curves do not overlap.

It is more convenient to count in ratio errors or percentage terms. The difference of counts in all cases is still 4%, but the error in the first case shall be 7/48 ~ 15%, in the second case – 23/500 ~ 4,6%, and only in the third case it shall become ~230/50000 ~ 0,5%, which is considerably less than the observed difference of 4%.

In conclusion I will open a “terrible secret”: what particular inferences, as I hoped, you would be able to suggest using our data on intensity (the number of counts per unit time) of appearance of cosmic rays. Everything should be done and understood / explained / interpreted in the same way according to that particular scheme which I have just exposed. We take and build the dependence of the intensity for any station (for instance station №3) cutting it into small intervals of one minute. In this way we shall receive a histogram – Fig. 5.

What is it like? Is that a real intensity of ray arrival? Is there jumping from minute to minute – from 0,03Hz to 0,25Hz (10 times difference)? Of course not! Simply, with the average one minute counting rate of ~ 0,12Hz there will be 7-8 counts. Smart children already know that with this average the error σ will be √7 ~ 2,6, and in ~ 70% of cases (more exact 68,3%) the obtained results shall be entitled to jump within the limits of 7± σ or from ~4 to ~10 counts a minute, and the frequency shall be from 0,07 to 0,17 Hz respectively.

Quite a different appearance shall have a week histogram of the same intensity with the number of cuttings of 36, that is, with averaging to 4,8 hours (Fig.6).

For this histogram the average in a bin for these 4,8 hours shall be estimated in ~ 2000 events with an error of ~ √2000 ~ 45. The ratio error shall be 45/2000 ~ 2,2%. For the average value of showers’ frequency of 0,12 Hz, the error of 2,2% shall be ~ 0,002Hz. Having reached such accuracy of measurement it may be stated that there is a growth of the EAS counting rate from 0,12Hz on February 05 to 0,13 Hz on February 09; and this difference makes 0,01 Hz±0,002. The difference of values for 5 errors shall look like an essential true effect and it shall require explanation. It would be most easy to assume that the EAS frequency has changed. But before we claim it we must show that we don’t see other mechanisms leading to a change of the EAS frequency. Let’s draw a line at this intriguing point. Try to think what other effect may bring to a change of the EAS registration counting rate and if there is any way to check that?

Thus, we have finished getting acquainted with another most important in actual science concept – assessment of a measurement error and how to estimate the level of the observed difference of two values (the criterion of the level of difference shall be limited to (± 3σ); to (± 5σ) would be even better.)

That’s why when you write your first reports (and further throughout your life) and make intelligent conclusions, then, please, try to do it in a way customary in science – if you make a statement taken from another person (from a text-book, or an article) – you should make a reference to this particular source. If you give the result of your own measurements which is the basic content of our project, then you are obliged to prove this statement is accurate.

For instance: over the specified period of the measurement the variable X under our research remained constant (it didn’t change) at the level of our measurements error and it amounted to 1,35 ±0,05.

I wish you good luck and am waiting for your thorough reports. By the way, my colleagues who have recently arrived have visited the neighboring Laboratory of Nuclear Reactions, where station № 5 was located and they eliminated the failure. As it turned out to be, it was a trigger that had gone out of operation and all the events were registered without preparatory selection according to coincidence of the signals in two our meters. If you look at the events of this station before the 27th of February and later, you will see yourselves what had changed.

Good luck to all of you,

See you later

G.A.