Tutorial: Actions

Compare Means

The CompareMeans class is ultimately getting at a tricky question in statistics. In English, we often think of this as whether separate distributions of data, like columns in a spreadsheet, are statistically significantly different from each other.

Consider an example. Suppose we have two columns of data in a spreadsheet. In the first column, the average is higher than the second. We can declare that and it might be true, but that data also has variation in it. Statistical tests at their core often try to balance 1) measures of centricity, 2) measures of dispersion, and 3) how much evidence is available. Intuitively, this makes sense. If the second group has a higher mean, but there is a lot of variation in the data and a small sample, this might indicate the result is less reliable than if there were more evidence or less variation. Statistical tests, while imperfect, try to balance these ideas and assign a number to them.

Quorum implements multiple parametric and non-parametric tests. It has the ability to measure differences in means or variables for data that is normally distributed and not normally distributed.

Listed below are functions that allow you to customize the statistical tests that you run. For example, the Ranked() function would be called when you want the results to be ranked or the call the RepeatedMeasures() function for tests that use repeated measures.

One Group

The tests listed below are used with a single sample group.

Compare Mean with a Given Value

Technical Test Name: One Sample T-Test

We can test whether the mean of a population is different from a proposed value in Quorum by using a One Sample T-Test.

Let's say you had a snack that claimed it had 25 grams of protein. You can compare whether it really does. To do this, collect a sample of the same energy bars from different stores and record the grams of protein in each one. Then run compare it to 25 to determine if it is true or not.

An example of how the test is implemented in Quorum is shown below.

Compare Ranked Median with a Given Value

Technical Test Name: Wilcoxon Signed Rank

Using a Wilcoxon Signed Rank Test, we can test the median of a group of numbers of a single sample with a given median for data that is not normally distributed to determine if it is different or not. Such data does not have a bell shaped curve or has really big values.

For example, you want to see if a new medicine influences driving abilities. You would randomly pick a group of drivers for your group that took the medicine and then compare their driving abilities to the basic driving abilities for drivers that have not taken any medicine.

An example of how the test is implemented in Quorum is shown below.

Two Groups: Different Subjects Per Group

The tests listed below are used with two sample groups from different populations.

Compare Means of Two Samples With Equal Variances

Technical Test Name: Two-Sample T-Test

We can use a Two-Sample T-test to determine if the means of two samples are equal assuming that the sample groups have equal variances or in other words, the data is spread out equally.

One example where this test could be used is comparing the average scores on a quiz for students from two different schools.

Below is an example of how to run it in Quorum

Compare Means of Two Samples With Unequal Variances

Technical Test Name: Welch's Two-Sample T-Test

We can test data samples that have unequal variances to determine if the means of two populations are equal through a Welch's Two-Sample T-Test. This is the default test in Quorum.

Suppose that we want to improve the reading ability of elementary students through a series of activities for reading. Two groups of students would be needed - one group participating in the reading activities and one group not participating. At the end of the trial period, all students take a test to see if their reading abilities have improved or not. We can then compare if the activities helped from all the student's scores.

To implement this test in Quorum, refer to the example below.

Compare Sample Populations

Technical Test Name: Mann-Whitney U-Test or Wilcoxon Rank-Sum Test

We can determine if two different independent samples are from the same population through a Mann-Whitney U-Test. This test is for data that is not normally distributed, so it doesn't have a bell shaped curve.

As an example, suppose we are trying to see if exercise or healthy eating is better at lowering sugar levels. One group would exercise and the other group would eat healthy to determine at the end of the study which one lowered sugar more.

To do this test in Quorum, refer to the example below.

Two Groups: Different Subjects Per Group

The tests listed below are used with two sample groups from different populations.

Compare Means of Two Dependent/Paired Groups with a Given Value

Technical Test Name: Paired Two-Sample T-Test

We can determine if the means of two paired groups are different from a proposed mean. To use this test in Quorum, refer to the example below.

One scenario for this test to be used would be seeing the effect of two different study methods on the average exam scores of students in the same class.

Below is an example of how to run it in Quorum

Compare Ranked Medians of Two Paired Samples With a Given Value

Technical Test Name: Wilcoxon Signed Rank Test - Two Groups

We use a Wilcoxon Signed Rank Test to determine if the ranked medians of two paired samples is different from a proposed median. It is the same as the one previously mentioned, just with two groups now.

For example, imagine you are a student in an 8th-grade science class, and your teacher wants to see if a new way of teaching helps students learn better. She decides to split the class into two groups - one group will be taught using the traditional method, and the other group will be taught using the new method. Both groups will take a pre-test to see what they already know, and then they will be taught using their assigned methods. After a certain period of time, both groups will take a post-test to see how much they have learned. The Wilcoxon Signed Rank Test can be used to compare the scores of the two groups. Since the same students are taking both tests, we can pair up their scores and see if there is a significant difference between the two groups.

To implement this test factorin Quorum, refer to the example below.

N Groups - Different Subjects Per Group

The tests listed below are used for a N (two or more) amount of different groups. Consider the post-hoc analysis tests after running the tests below.

Compare Means of Two or More Different Groups with Equal Variances

Technical Test Name: One-Way ANOVA

We can check if the means of two or more different groups are different from each other by using a One-Way ANOVA. It would work best with three or more groups.

A scenario in which this test would be used is finding the difference between average headache relief given three different medicines.

An example of how to implement this test in Quorum is shown below.

We can use Tukey's HSD Multiple Comparison Test as a followup to do some post-hoc analysis.

Compare Means of Two or More Different Groups with Unequal Variances

Technical Test Name: Welch's ANOVA

We can use Welch's ANOVA Test to check if the means of two or more different groups with unequal variances are different from each other.

One scenario to use this test would be determining how outgoing different hair colors (brown, blonde, black, or red) are. It does not assume variances of the groups are equal meaning that there might be data that varies from the average.

An example of how to implement this test in Quorum is shown below.

The Games-Howell Multiple Comparison Test is most commonly used as the post-hoc analysis for this test.

Compare Medians of Two or More Different Groups

Technical Test Name: Kruskal-Wallis H-Test

We use a Kruskal-Wallis Test to compare the medians of two or more different groups to see if they are different. It is an ANOVA test for data that does not have a normal distribution.

Suppose we want to measure the growth of a plant in different sunlight. We plant some seeds in five different spots with either lots of sunlight directly, medium sunlight, low sunlight, sunlight through a window, or no sunlight. After some time, we can measure the height of the plants that grew in each different sunlight. A Kruskal-Wallis test will help determine if sunlight impacts the growth of the plant using the median.

An example of how to do this test in Quorum is shown below.

N Groups - Same Subjects Per Group (Repeated Measurements/Observations)

The tests listed below are used with a N (two or more) amount of groups from the same population.

Compare Means of Several Similar Groups

Technical Test Name: Repeated Measures ANOVA Test

We can determine if the means of several similar groups are the same or not through repeated observations or trials over time.

One scenario to use the Repeated Measures One-Way ANOVA test would be tracking the weight gain of a puppy after eating a specific dog food for 6 weeks. Weighing the puppy before starting the dog food and every week after that for 6 weeks would be the repeated measures part.

Below is an example of how to run it in Quorum

We can run a Bonferroni Pairwise Procedure as a post hoc analysis test.

Compare Medians of Several Similar Groups

Technical Test Name: Friedman Test

We use a Friedman Test to compare the difference between the medians of three or more groups through repeated observations.

An example scenario of when this test could be used is watching movies. We could have a group of people watch five different movies and rate how good it was. Every person will rate the five movies so a Friedman test could determine if there are any major differences in the ratings of the five movies.

An example of how to do this test in Quorum is shown below.

Factorial ANOVA and MANOVA

A Factorial ANOVA is a type of ANOVA that looks at two or more factors at the same time and can be either balanced or unbalanced. A factor is just a category or group that we're interested in.

This test uses the factors that we are interested in and tries to see what changes they caused on one dependent variable. For example, let's say we want to know if there's a difference in test scores based on both the number of hours studied per day AND the type of study method used (flashcards, reading notes, or practicing problems). We could use a Factorial ANOVA to analyze this data and see if there are significant differences between the different groups. Basically, a Factorial ANOVA is a statistical test that looks at multiple factors at the same time and can either be balanced or unbalanced which is explained in more detail below.

The tests listed below are used for a N (two or more) amount of different groups with more than one factor.

Compare Several Means With Same Number of Groups

Technical Test Name: Balanced Factorial ANOVA

In a Balanced Factorial ANOVA, each group has the same number of participants.

We are going to use the example above to see what different study methods and different amounts of time affect test scores. For a Balanced Factorial ANOVA, if we have 3 different study methods and 3 different study hours (which are the factors), we would need the same number of participants in each of the 9 resulting groups. Then it would make the data balanced and we can then compare any differences between the groups.

Compare Several Means With Different Number of Groups

Technical Test Name: Unbalanced Factorial ANOVA

In an Unbalanced Factorial ANOVA, groups might have more participants than others. This happens mostly when there are not enough groups or not enough people to recruit.

Let's use the example above to see what different study methods and different amounts of time affect test scores. We might have three groups for studying time: one group that studies for 1 hour, another group that studies for 2 hours, and a third group that studies for 3 hours. With these, there might be more participants in one group than in another like we have 10 participants in the 1 hour group, 15 participants in the 2 hour group, and 8 participants in the 3 hour group. This unequal grouping makes it harder to see differences between the groups, but it can be done using an Unbalanced Factorial ANOVA!

Factorial MANOVA

MANOVA stands for multivariate analysis of variance which is a type of statistical analysis that looks at the relationship between multiple variables.

A Factorial MANOVA looks at the effects of multiple independent variables (factors) on multiple dependent variables at the same time. This test helps us compare groups of data based on different variables (factors or characteristics) to see if there are any differences between them. Our goal is to see how the independent variables or factors affect the dependent variables.

The tests listed below are used for a N (two or more) amount of different groups with more than one factor.

Compare Means Of Several Groups With Multiple Dependent and Independent Variables

Technical Test Name: Factorial MANOVA

In a Balanced Factorial ANOVA, each group has the same number of participants.

Imagine you have a bunch of data about different things, like the height, weight, and age of a group of people. Now, you want to find out if certain factors, like gender and exercise, have an effect on people's height or weight. A Factorial MANOVA lets you look at multiple factors together at the same time instead of looking at them one at a time.

Next Tutorial

In the next tutorial, we will discuss Compare Distributions, which describes how to check the distributions of data.