Abstract
This chapter introduces permutation methods for two-sample tests. Included in this chapter are six example analyses illustrating computation of exact permutation probability values for two-sample tests, calculation of measures of effect size for two-sample tests, the effect of extreme values on conventional and permutation two-sample tests, exact and Monte Carlo permutation procedures for two-sample tests, application of permutation methods to two-sample rank-score data, and analysis of two-sample multivariate data. Included in this chapter are permutation versions of Student’s two-sample t test, the Wilcoxon–Mann–Whitney two-sample rank-sum test, Hotelling’s multivariate T 2 test for two independent samples, and a permutation-based alternative for the four conventional measures of effect size for two-sample tests: Cohen’s \(\hat{d}\), Pearson’s r 2, Kelley’s 𝜖 2, and Hays’ \(\hat{\omega }^{2}\).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For two treatments a fair coin works quite well with heads and tails. For three treatments, a fair die is often used with faces with one or two pips assigned to the first treatment, faces with 3 or 4 pips assigned to the second treatment, and faces with 5 or 6 pips assigned to the third treatment. For four treatments, a shuffled deck of cards works well with clubs (♣), diamonds (♢), hearts (♡), and spades (♠) assigned to Treatments 1, 2, 3, and 4, respectively.
- 2.
In some disciplines tests on two independent samples are known as between-subjects tests and tests for two dependent or related samples are known as within-subjects tests.
- 3.
Also see a discussion by S.M. Stigler in The Seven Pillars of Statistical Wisdom [14, pp. 91–92].
- 4.
In 2017 the average student debt for law-school graduates was reported to be $141,000 and the average student debt for medical-school graduates was reported to be $192,000.
- 5.
Degrees of freedom are not relevant for any nonparametric, distribution-free statistic. However, it is noteworthy that degrees of freedom may be required for a test statistic that is nonparametric but is not distribution-free, such as Pearson’s χ 2 test statistics for goodness of fit and independence.
- 6.
When fitting a continuous mathematical function, such as the normal probability distribution, to a discrete permutation distribution, it is oftentimes necessary to correct the fit by adding or subtracting 0.5 to compensate for the discreteness of the distribution.
References
Barnard, G.A.: 2 × 2 tables. A note on E. S. Pearson’s paper. Biometrika 34, 168–169 (1947)
Cohen, J.: Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol. Bull. 70, 213–220 (1968)
Feinstein, A.R.: Clinical biostatistics XXIII: the role of randomization in sampling, testing, allocation, and credulous idolatry (Part 2). Clin. Pharmacol. Ther. 14, 898–915 (1973)
Hotelling, H., Pabst, M.R.: Rank correlation and tests of significance involving no assumption of normality. Ann. Math. Stat. 7, 29–43 (1936)
Johnston, J.E., Berry, K.J., Mielke, P.W.: A measure of effect size for experimental designs with heterogeneous variances. Percept. Motor Skill. 98, 3–18 (2004)
Johnston, J.E., Berry, K.J., Mielke, P.W.: Permutation tests: precision in estimating probability values. Percept. Motor Skill. 105, 915–920 (2007)
Kendall, M.G., Babington Smith, B.: On the method of paired comparisons. Biometrika 31, 324–345 (1940)
Macdonell, W.R.: On criminal anthropometry and the identification of criminals. Biometrika 1, 177–227 (1902)
Maxwell, S.E., Camp, C.J., Arvey, R.D.: Measures of strength of association: a comparative examination. J. Appl. Psychol. 66, 525–534 (1981)
McHugh, R.B., Mielke, P.W.: Negative variance estimates and statistical dependence in nested sampling. J. Am. Stat. Assoc. 63, 1000–1003 (1968)
Mielke, P.W., Berry, K.J., Johnson, E.S.: Multi-response permutation procedures for a priori classifications. Commun. Stat. Theor. Methods 5, 1409–1424 (1976)
Scott, W.A.: Reliability of content analysis: the case of nominal scale coding. Public Opin. Quart. 19, 321–325 (1955)
Spearman, C.E.: ‘Footrule’ for measuring correlation. Brit. J. Psychol. 2, 89–108 (1906)
Stigler, S.M.: The Seven Pillars of Statistical Wisdom. Harvard University Press, Cambridge (2016)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Berry, K.J., Johnston, J.E., Mielke, P.W. (2019). Two-Sample Tests. In: A Primer of Permutation Statistical Methods. Springer, Cham. https://doi.org/10.1007/978-3-030-20933-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-20933-9_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20932-2
Online ISBN: 978-3-030-20933-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)