Initiation à la méthodologie statistique

Abstract

Statistical analysis does not absolutely need very large samples to draw a valuable conclusion. It is not only calculation, but above all reflexion about methodology, and calculation itself is only a small part of it. It is the way to get the best information from the available data.

The basic question is the following: what is the probability of drawing a given number of white balls from a box with 50 white and 50 black balls ? In 95% of the cases, this number will be between 2 and 8: this is the confidence interval with a 5% error level, which is the common significance level. This interval will be closer if the number of balls drawn increases: that is why large numbers of subjects are interesting on the statistical point of view.

On the contrary, if 4 white and 6 black balls are drawn from a 100 ball box, can the proportion of white balls in this box be estimated ? With a 5% error level, the proportion will be between 10 and 70%: this is the confidence interval with a 5% error level. As before, this interval will be closer if the number of balls drawn increases.

The common question is the comparison of two or more results: it follows the same but repeated principle. From a first 1.000 ball box are drawn 40 white and 60 black balls; from a second 1.000 ball box 60 white and 40 black balls are drawn: are the proportion of white balls different in both boxes? The confidence interval with a 5% error level for the first box is between 30 and 50%; the confidence interval with a 5% error level for the second box is between 50 and 70%. As these intervals have a common value, it is impossible to simply conclude a difference in composition. Statistical tests (Chi2 test) allow detection of more subtle differences: in this example the Chi2 test compares the result of a mathematical calculation to a theoretical value: the difference is significant (with a 5% error level) if the calculated value is over the theoretical one, then it can be concluded that the proportion of white balls are different in both boxes. It is important to notice that the statistical tests not only study means or percentages, but the actual number for each studied subject: as before, the test will detect a more subtle difference if the number of subjects increases.

There are two risks of error during statistical calculations. a is the risk of drawing a false-positive conclusion: it is always under 5%. b is the risk of not finding an actual difference: it is the false-negative risk. Both risks are related to the number of subjects studied and to the smallest difference by a mathematical calculation. On the contrary this calculation let us know how many subjects have to be studied with given a and b risks. It is important to understand that the results of a statistical test only give a relation, that is an association between two observed results, but not always a correlation, that is a consequence of the observed results. The correlation can only be studied if the studied groups are comparable.

Statistical calculation is only one step of the whole statistical methodology, which involves seven steps: to ask the question precisely, to define the accepted error level and the nescessary number of subjects; to define the study design, and to obtain comparability between the different subgroups: this can be better done by randomisation, which allows separation of prognostic factors at random in all subgroups studied; to choose the criteria for results; to collect the data; to choose the appropriate statistical test: each test has precise conditions of application, which have to be respected; to apply the test and to perform the mathematical calculation; to draw the conclusion.

These basic principles have to be respected for a confident statistical study:

• - the methodology is the most important point and shoud be perfect;

• - the study must be designed before starting;

• - the computer and software are only helpful but do not work alone;

• - the correlation can only be studied if the subgroups are comparable;

• - the comparability is better controlled by randomisation.

Résumé

La statistique n’est pas la loi des grands nombres : un effectif important n’est pas indispensable pour tirer une conclusion statistique valable. La statistique n’est pas non plus que du calcul : c’est avant tout de la réflexion méthodologique, et le calcul mathématique n’occupe qu’une part infime de la statistique. La statistique est en fait la façon d’exploiter au mieux les données dont on dispose.

Le calcul statistique n’est qu’une petite étape de la méthodologie statistique prise dans son ensemble. On peut définir sept étapes dont le respect et la chronologie sont indispensables pour pouvoir tirer les conclusions de façon irréfutable : 1) poser la question de façon précise, en définissant les risques d’erreur acceptés, la différence recherchée et le nombre de sujets étudiés ; 2) mettre en place le déroulement de l’étude, et en particulier assurer la comparabilité des différents groupes entre eux ; 3) choisir les critères d’étude des résultats ; 4) saisir les données ; 5) choisir le test statistique adapté à la situation ; 6) effectuer le calcul statistique (c’est la seule place de l’ordinateur !) ; 7) tirer la conclusion.

La statistique n’est pas du calcul, mais de la méthodologie qui ne souffre pas la médiocrité. Malgré les croyances répandues, l’ordinateur et le logiciel de statistique ne font pas le statisticien.