Non-Parametric Multi-Comparison Demo

The StatsNPMCTest operation supports a number of comparison tests. The following examples illustrate the tests and the optional flags.

1. Tukey type test analysing the data in the following 5 waves:

data1data2data3data4data5
36.3458.5550.4241.4147.81
52.085740.0464.2146.28
50.0944.941.7758.3359
37.8152.6642.6160.3755.73
58.9867.9937.7555.3254.58
63.9352.9841.4255.7543.74
37.5551.139.4863.7459.02
40.6447.9643.7967.8646.75

To run the test execute the command:

StatsNPMCTest/T=1/Q/TUK data1,data2,data3,data4,data5

The results are displayed in the NP Multiple Comparison (Tukey) table:

PairDifferenceSEqqcConclusion
data4_vs_data316333.06564.92963.857660
data4_vs_data111633.06563.508183.857661
data4_vs_data56133.06561.844823.857661
data4_vs_data24033.06561.209723.857661
data2_vs_data312333.06563.719883.857661
data2_vs_data17633.06562.298463.857661
data2_vs_data52133.06560.6351013.857661
data5_vs_data310233.06563.084783.857661
data5_vs_data15533.06561.663363.857661
data1_vs_data34733.06561.421423.857661

The Tukey test conclusions indicate that at the 0.05 significance level only they hypothesis that the mean of data4 equals the mean of data3 is rejected. All other means are taken to be the same. If we increase the significance level to 0.1 we have:

StatsNPMCTest/T=1/Q/TUK/ALPH=0.1 data1,data2,data3,data4,data5

The results are displayed in the table:

PairDiffSEqqcConclusion
data4_vs_data316333.06564.92963.478280
data4_vs_data111633.06563.508183.478280
data4_vs_data56133.06561.844823.478281
data4_vs_data24033.06561.209723.478281
data2_vs_data312333.06563.719883.478280
data2_vs_data17633.06562.298463.478281
data2_vs_data52133.06560.6351013.478281
data5_vs_data310233.06563.084783.478281
data5_vs_data15533.06561.663363.478281
data1_vs_data34733.06561.421423.478281

In this case H0 has to be rejected also when comparing data4 with data1 and when comparing data2 with data1.

2. Student-Newman-Keuls test example.

This is a non-parametric variation on the Student-Newman-Keuls test where the standard error is a function of the parameter p (the rank difference). The test requires that all input waves have the same number of points. To run the test execute:

StatsNPMCTest/T=1/Q/SNK data1,data2,data3,data4,data5

The results are displayed in the NP Multiple Comparison (SNK) table:

PairDiffSEpqpqcConclusion
data4_vs_data316333.065654.92963.857660
data4_vs_data111626.53344.371913.633160
data4_vs_data5612033.053.314491
data4_vs_data24013.46622.970442.771810
data2_vs_data312326.53344.635743.633160
data2_vs_data1762033.83.314490
data2_vs_data52113.46621.559482.771811
data5_vs_data31022035.13.314490
data5_vs_data15513.46624.084362.771810
data1_vs_data34713.46623.490272.771810

In this test the statistic q depends on the value of p and therefore differences between waves tend to be amplified relative to the Tukey test above.

3. Multi-Comparison when waves do not have the same number of points

The Student-Newman-Keuls test above requires that all waves have the same number of points. The Dunn-Holland-Wolfe test supports unequal number of points as well as ties in ranks. In this example we analyze the following waves:

data2data3data4data5data6data7
58.5550.4241.4147.8138.1750.55
5740.0464.2146.2836.9649.63
44.941.7758.335938.2849.57
52.6642.6160.3755.7333.1353.92
67.9937.7555.3254.5839.9754.4
52.9841.4255.7543.7447.9939.62
51.139.4863.7459.0249.66
47.9643.7967.8646.7542.51

To run the test execute the command:

StatsNPMCTest/T=1/Q/DHW data2,data3,data4,data5,data6,data7

The results are displayed in the NP Multiple Comparison (DHW) table:

PairDiffSEQQcConclusion
data4_vs_data626.56.711313.948562.93520
data4_vs_data324.3756.711313.631932.93520
data4_vs_data712.57.249041.724372.93521
data4_vs_data58.8756.711311.322392.93521
data4_vs_data25.6256.711310.8381382.93521
data2_vs_data620.8756.711313.110422.93520
data2_vs_data318.756.711312.793792.93521
data2_vs_data76.8757.249040.9484012.93521
data2_vs_data53.256.711310.4842572.93521
data5_vs_data617.6256.711312.626162.93521
data5_vs_data315.56.711312.309532.93521
data5_vs_data73.6257.249040.5000662.93521
data7_vs_data6147.249041.931292.93521
data7_vs_data311.8757.249041.638152.93521
data3_vs_data62.1256.711310.316632.93521

In this case, the test indicates that we should reject H0 (equality of means) for the pairs data4 and data6, data4 and data3, data2 and data6.

4. Multi-Comparison to a control.

Using the waves data1-data6 above, we can test the hypothesis of equal means (default /TAIL) of each wave compared with the control wave designated as data4. To execute the test, select the blue line below and type Ctrl-Enter:

StatsNPMCTest/T=1/Q/CIDX=3 data1,data2,data3,data4,data5,data6

The results are displayed in NP Multiple Comparison to Control table:

PairDiffSEqq'conclusion
data4_vs_data6205563.660712.511460
data4_vs_data3183563.267862.511460
data4_vs_data1135562.410712.511461
data4_vs_data566561.178572.511461
data4_vs_data241560.7321432.511461

The test concludes that H0 (equal means) has to be rejected for the wave data3 and the wave data6 when compared with the control wave data4.

We can modify the test for H0: μc≤μa using /TAIL=1. Here μc represents the mean of the control wave while μa is the mean of any other wave in the input group.

StatsNPMCTest/T=1/Q/CIDX=3/TAIL=1 data1,data2,data3,data4,data5,data6
PairDiffSEqq'Conclusion
data4_vs_data6205563.660712.233820
data4_vs_data3183563.267862.233820
data4_vs_data1135562.410712.233820
data4_vs_data566561.178572.233821
data4_vs_data241560.7321432.233821

Clearly the mean of the control wave is greater than the means of the waves data1, data3 and data6.

Testing for the other possible tail (H0: μc≥μa ):

StatsNPMCTest/T=1/Q/CIDX=3/TAIL=2 data1,data2,data3,data4,data5,data6

The results are displayed in the table:

PairDiffSEqq'Conclusion
data4_vs_data6205563.660712.233821
data4_vs_data3183563.267862.233821
data4_vs_data1135562.410712.233821
data4_vs_data566561.178572.233821
data4_vs_data241560.7321432.233821

5. Multiple Contrasts

Suppose we wanted to test the hypothesis H0: μ123456.

You can create the contrast wave using the command:

Make/O/N=6 contrastW={1,1,1,2,2,2}

To run the test execute:

StatsNPMCTest/T=1/Q/CONW=contrastW data1,data2,data3,data4,data5,data6

The results are displayed in the NP Multiple Contrasts table (shown here transposed):

Contrast3.66667
SE4.04145
S0.907265
Critical_Chai3.32724
Conclusion1