Spearman Rank Correlation [Simply explained]

Summary

The video script explains the concept of Spearman's rank correlation, a non-parametric method to measure the relationship between two variables using their ranks instead of raw data. It contrasts this with Pearson correlation and demonstrates the process through an example involving reaction times and ages of computer players. The script also covers how to calculate Spearman's correlation coefficient, interpret its strength and direction, and use a t-test to determine if the correlation is significantly different from zero, as illustrated with a dataset provided in the video description.

Takeaways

• 📊 Spearman correlation is a non-parametric statistical measure used to assess the strength and direction of the relationship between two variables.
• 🔄 Unlike Pearson correlation, Spearman correlation does not use the raw data but instead uses the ranks of the data.
• 🎯 In the example provided, reaction time and age of computer players are ranked to calculate Spearman correlation.
• 📈 The Spearman rank correlation coefficient is calculated using the ranks of the data, similar to the Pearson correlation but with ranks.
• 📉 The Spearman correlation coefficient, 'rs', ranges from -1 to 1, indicating the strength and direction of the relationship between variables.
• 🔢 The formula for Spearman correlation involves summing the squares of the differences in ranks (D) and is applicable when there are no rank ties.
• 📝 The correlation coefficient can be used to interpret the strength of the relationship: negative values indicate a negative correlation, positive values a positive correlation, and zero indicates no correlation.
• ✅ A hypothesis test can be conducted to determine if the correlation coefficient is significantly different from zero, using a t-test and comparing the p-value to a significance level.
• 📉 The null hypothesis states that there is no relationship (correlation coefficient equals zero), while the alternative hypothesis suggests there is a relationship.
• 📊 The significance of the correlation can be assessed using a t-test statistic, where a p-value less than the significance level (commonly 0.05) leads to the rejection of the null hypothesis.
• 🔗 Data from the example in the script yields a p-value of 0.002, which is less than 0.05, indicating a significant correlation and rejection of the null hypothesis.

Q & A

What is the Spearman correlation?

The Spearman correlation, also known as Spearman's rank correlation, is a non-parametric measure of rank correlation that assesses how well the relationship between two variables can be described using a monotonic function.

How does Spearman correlation differ from Pearson correlation?

While both Spearman and Pearson correlations measure the strength and direction of the relationship between two variables, Spearman correlation uses the ranks of the data instead of the raw data, making it a non-parametric alternative to the Pearson correlation which requires normally distributed data.

What is the process of calculating Spearman rank correlation?

To calculate the Spearman rank correlation, first assign ranks to the data points for each variable. Then, calculate the difference in ranks for each pair of data points, square these differences, sum them up, and divide by the number of observations to find the Spearman correlation coefficient.

How are ranks assigned to the data in Spearman correlation?

Ranks are assigned based on the order of the data points. The smallest value gets rank 1, the next smallest gets rank 2, and so on. If there are ties, assign the average rank to the tied values.

What does a Spearman correlation coefficient value of 0.9 indicate?

A Spearman correlation coefficient of 0.9 indicates a very strong positive relationship between the two variables, meaning that as one variable increases, the other tends to increase as well.

What is the range of the Spearman correlation coefficient?

The Spearman correlation coefficient ranges from -1 to 1, where -1 indicates a perfect negative relationship, 1 indicates a perfect positive relationship, and 0 indicates no relationship.

How can you determine if a Spearman correlation coefficient is statistically significant?

To determine if a Spearman correlation coefficient is statistically significant, you can use a t-test. If the calculated p-value is less than the significance level (commonly 0.05), the null hypothesis that there is no relationship is rejected.

What is the null hypothesis in a Spearman correlation test?

The null hypothesis in a Spearman correlation test is that the correlation coefficient (ρ) is equal to zero, indicating no relationship between the two variables.

What is the alternative hypothesis in a Spearman correlation test?

The alternative hypothesis in a Spearman correlation test is that the correlation coefficient (ρ) is not equal to zero, indicating that there is a relationship between the two variables.

Can the Spearman correlation be used with non-normally distributed data?

Yes, the Spearman correlation is particularly useful with non-normally distributed data or ordinal data because it does not assume normality and is based on the ranks of the data rather than the actual values.

How does the presence of outliers affect the Spearman correlation?

Since Spearman correlation is based on ranks, it is less affected by outliers compared to the Pearson correlation, which is sensitive to extreme values in the data.

Outlines

00:00

📊 Understanding Spearman's Rank Correlation

The first paragraph introduces Spearman's rank correlation, a non-parametric statistical method used to measure the relationship between two variables. Unlike Pearson correlation, which uses raw data, Spearman's method employs the ranks of the data. The paragraph provides an example involving the reaction time of computer players and their ages, demonstrating how to assign ranks to the data and how to calculate the Spearman correlation coefficient. It also explains that the Spearman correlation can yield the same result as the Pearson correlation when there are no ties in the ranks. The formula for calculating Spearman's correlation without ties is given, and the interpretation of the correlation coefficient's value is discussed, including how to determine the strength and direction of the correlation.

05:04

🔍 Hypothesis Testing with Spearman Correlation

The second paragraph delves into hypothesis testing using the Spearman correlation coefficient. It explains the process of testing whether the correlation in the sample data is significantly different from zero, which would indicate a relationship in the population. The paragraph outlines the null hypothesis (no relationship, correlation coefficient equals zero) and the alternative hypothesis (there is a relationship). It also describes how to use a t-test to determine if the correlation coefficient is significantly different from zero, with the significance level typically set at 5%. The example given in the paragraph results in a p-value of 0.002, which is less than the 0.05 threshold, leading to the rejection of the null hypothesis. The paragraph concludes by encouraging viewers to download the dataset for further exploration and thanking them for watching the video.

Keywords

💡Spearman correlation

Spearman correlation, also known as Spearman's rank correlation, is a non-parametric measure of rank correlation that assesses how well the relationship between two variables can be described using a monotonic function. In the video, it is contrasted with Pearson correlation and is used to examine the relationship between a player's reaction time and their age, with the correlation coefficient indicating the strength and direction of the relationship.

💡Pearson correlation

Pearson correlation is a statistical measure that expresses the linear relationship between two continuous variables. The script explains that while Pearson correlation uses the raw data values, Spearman correlation uses the ranks of the data, making it a non-parametric alternative. The video provides an example where both methods are applied to the same data set, yielding the same correlation coefficient of 0.9.

💡Non-parametric

Non-parametric refers to statistical methods that do not assume a specific distribution for the underlying populations. In the context of the video, Spearman correlation is described as the non-parametric counterpart of Pearson correlation, meaning it does not require the data to follow a normal distribution and can be used when the assumptions of parametric tests are not met.

💡Ranks

Ranks in statistics refer to the order of data when arranged in ascending or descending order. In the video, ranks are assigned to both reaction time and age of the players. The use of ranks is central to calculating the Spearman correlation, as it allows for the assessment of the relationship between variables without relying on the actual data values.

💡Scatter plot

A scatter plot is a type of plot that uses Cartesian coordinates to display values for two variables for a set of data. In the video, a scatter plot is used to visualize the raw data of age and reaction time, and then again to represent the ranks of these variables, which is crucial for understanding how Spearman correlation is derived from the ranked data.

💡Correlation coefficient

The correlation coefficient is a measure that expresses the extent to which two variables are linearly related. In the video, both the Spearman and Pearson correlation coefficients are calculated to be 0.9, indicating a strong positive relationship between the variables. The script explains how this coefficient varies between -1 and 1, with the sign indicating the direction of the relationship.

💡Hypothesis testing

Hypothesis testing is a statistical method used to determine whether there is enough evidence to support a claim. In the video, the null hypothesis is that there is no relationship (correlation coefficient equals zero), and the alternative hypothesis is that there is a relationship. The script describes how a t-test is used to calculate a p-value, which helps decide whether to reject or fail to reject the null hypothesis.

💡Significance level

The significance level, often denoted as alpha, is the threshold used in hypothesis testing to determine if the results are statistically significant. In the video, a significance level of 5% is mentioned, meaning if the p-value is less than 0.05, the null hypothesis is rejected, indicating a statistically significant correlation.

💡P-value

The p-value is the probability of obtaining results at least as extreme as the observed results, assuming the null hypothesis is true. In the context of the video, a p-value of 0.002 is calculated, which is less than the significance level, leading to the rejection of the null hypothesis and supporting the existence of a correlation.

💡Data set

A data set is a collection of data, typically used for analysis. The video script mentions a data set that can be downloaded for further analysis, which includes the reaction times and ages of computer players. This data set is used to calculate and compare the Spearman and Pearson correlation coefficients.

💡T-test

A t-test is a statistical test used to compare the means of two groups to determine if there is a significant difference between them. In the video, a t-test is used to evaluate the significance of the Spearman correlation coefficient calculated from the sample data, helping to determine if the correlation is significantly different from zero.

Highlights

Spearman correlation is a non-parametric method to examine the relationship between two variables.

Unlike Pearson correlation, Spearman correlation uses the ranks of data rather than the raw data.

The calculation of Spearman correlation involves assigning ranks to each data point for the variables being studied.

An example is given where the reaction time and age of computer players are measured and ranked.

The ranks are used to form a scatter plot for the Spearman correlation analysis.

Spearman and Pearson correlation coefficients can yield the same result when calculated from ranks.

The Spearman correlation coefficient is calculated using the formula that involves the sum of squared differences in ranks.

The Spearman correlation coefficient ranges from -1 to 1, indicating the strength and direction of the relationship.

A table is used to interpret the strength of the correlation based on the coefficient's value.

The significance of the correlation coefficient can be tested using a t-test against the null hypothesis of no relationship.

A p-value is calculated to determine if the correlation coefficient is significantly different from zero.

If the p-value is less than the significance level, the null hypothesis is rejected, indicating a significant relationship.

The example provided calculates a p-value of 0.002, which is less than 0.05, leading to the rejection of the null hypothesis.

Data can be downloaded for further analysis, with the link provided in the video description.

The video concludes by summarizing the process and significance of calculating and interpreting the Spearman correlation.

Transcripts

what is a Spearman correlation

spearman's rank correlation examines the

relationship between two variables isn't

that exactly what the Pearson

correlation does that's right the

Spearman rank correlation is the

non-parametric counterpart of the

Pearson correlation but there is an

important difference between both

correlation coefficients Spearman

correlation does not use the raw data

but the ranks of the data let's look at

this with an example we measure the

reaction time of 8 computer players and

ask their age when we calculate a

Pearson correlation we simply take the

two variables reaction time and age and

calculate the Pearson correlation

coefficient however we now want to

calculate the Spearman rank correlation

so first we assign a rank to each person

for reaction time and age the reaction

time is already sorted by size 12 is the

smallest value so gets rank 1 15 the

second smallest so gets Rank 2 and so on

and so forth we are now doing the same

with age here we have the smallest value

there the second smallest value the

third smallest value fourth smallest and

so on and so forth let's take a look at

this in a scatter plot here we see the

raw data of age and reaction time but

now we would like to use the rankings so

we form ranks from the variables age and

reaction time

through this transformation we have now

distributed the data more evenly to

calculate spiem and correlation now we

simply calculate the P using correlation

from the ranks

so the Spearman correlation is equal to

the Pearson correlation only that the

ranks are used instead of the raw values

let's have a quick look at that in data

tab here we have the reaction time and

age and there we have the just created

ranks of the reaction time and age now

we can either calculate the spement

correlation of the reaction time and age

where we get a correlation of 0.9 or we

can calculate the Pearson correlation

from the ranks there we also get 0.9 so

exactly the same as before if you like

you can download the data set you can

find the link in the video description

if there are no rank ties we can also

use this equation to calculate the

Spearman correlation n is the number of

cases and D is the difference in ranks

between the two variables referring to

our example we get the different D's

with this one minus 1 which is 0 2 minus

three is minus one three minus two is

one and so on now we Square the

individual D's and add them all up

so the sum of d i squared is eight n

which is the number of people is also 8

in this example if you put everything in

we get a correlation coefficient of 0.9

just like the Pearson correlation

coefficient R the Spearman correlation

coefficient r s also varies between

-1 and 1 with the help of the

coefficient we can now determine two

things number one how strong the

correlation is and number two in which

direction the correlation goes the

strength of the correlation can be read

in a table if we have a coefficient

between

-1 and less than zero there is a

negative correlation thus a negative

relationship between the variables if we

have a coefficient between greater than

zero and one there is a positive

correlation that is a positive

relationship between the two variables

if the result out is zero we have no

correlation often however starting from

a sample we want to test a hypothesis

about the population we calculated the

correlation coefficient from the sample

data and now we can check if the

correlation coefficient is significantly

different from zero thus the null

hypothesis is the correlation

coefficient R is equal to zero there is

no relationship and the alternative

hypothesis is the correlation

coefficient R is n equal to zero there

is a relationship whether the

correlation coefficient is significantly

different from zero based on the sample

collected can be checked using a t-test

where R is the correlation coefficient

and N is the sample size a p-value can

then be calculated from the test

statistic T if the p-value is less than

the specified significance level which

is usually five percent then the null

hypothesis is rejected otherwise it is

not if you use data depth for the

calculation of the example we get a

p-value of

0.002 the p-value is therefore smaller

than 0.05 and we can therefore reject

the null hypothesis that in the

population the correlation coefficient

is zero thanks for watching and I hope

you enjoyed the video