{'fraction_non_alphanumeric': 0.02842497670083877, 'fraction_numerical': 0.015610438024231128, 'mean_word_length': 4.678571428571429, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Mann–Whitney U test when variances are unequal

The Mann–Whitney U test is one of the most widely used statistical tests in studies of behaviour. In 1999, 42 of 151 articles surveyed (27.8%) in volume 56 of Animal Behaviour used this test, and another eight used its extension to more than two samples: the Kruskal–Wallis test. The Mann–Whitney U test, like many other nonparametric tests based on rank, assumes equal variances in the two populations (in this commentary, I use the term population only in the statistical sense) from which the two samples being compared are taken. For example, Siegel & Castellan (1988, page 137) noted that ‘we are implying that variability or variances of the distributions are equal’. However, authors usually use the Mann–Whitney U test without examining whether the variances are equal. For 21 of the 50 articles in volume 56 of Animal Behaviour (1999), that used a U test or Kruskal– Wallis test the difference between the variances can be tested (I did not calculate the variance when the variation, for example SE, was given only by error bars in figures). The variances were significantly different at the 5% level in 12 of 21 papers. Another three noted that the U test or Kruskal–Wallis test was used when the variances were unequal. These results suggest that the U test or Kruskal–Wallis test is often used when the variances are unequal in behavioural studies. In this commentary, I show that the type 1 error rate of the Mann–Whitney U test inflates under unequal variance (heteroscedasticity) and suggest some solutions. Consider two populations in which the means are the same but the variances are different. Since the means are the same, when the difference of the means (location, to be exact) is tested by the two samples from the two populations, the probability that a statistical test erroneously indicates a significant difference should be equal to or less than the nominal (declared) level of significance (e.g. 0.05). This is a requirement for statistical tests in general. I determined this probability, the type 1 error rate, for the U test by computer simulation. Two samples of data were generated by random numbers from two distributions with the same mean but different variances. Note that, since the variances are those of populations, analysts are assumed not to know them. Both of the distributions were normal or uniform. The simulations were repeated 10 000 times for a given condition (the distribution of populations, sample sizes and ratio of standard deviations of the two populations). Table 1 shows that the type 1 error rate of the U test depends on the difference between two variances and inflates for a wide range of large differences between the variances. The results are similar for both the normal and uniform distributions. Table 1 also shows the asymmetrical effect of sample sizes. When a small sample is taken from a population with a small variance, the unequal variances cause the type 1 error rate of the U test to be smaller than the nominal value, in contrast to other cases. This can be explained by the combination of a small sample from the population with the small variance and a large one from the population with the large variance making a large difference between locations of samples unlikely. The result shown in Table 1 also suggests that the power of the U test is lower in this case. The result shows inflation of the type 1 error rate of the U test when the variances differ. One may wonder whether the U test is nonparametric and thus distribution free. The U test (and many other nonparametric tests) assumes that two samples are taken from a common population when the null hypothesis is true. The distribution of the common population can be any type. Thus, this test is distribution free. However, taking two samples from one population means that the variances of the two populations are assumed to be equal (e.g. Siegel & Castellan 1988). Hayes (2000) pointed out a similar problem for some randomization tests. Since the U test is not appropriate when the variances are different, a test procedure valid under this situation is Correspondence: E. Kasuya, Department of Biology, Kyushu University, Hakozaki, Fukuoka, 812-8581, Japan (email: ekasuscb@mbox.nc.kyushu-u.ac.jp)