Convergent and Divergent Validity description and analysis

Summary

Dr. Dan discusses convergent and divergent validity, explaining their definitions, testing methods, and interpretation of results. Convergent validity is shown when items within the same factor correlate with effect sizes between 0.30 and 0.70, indicating they're measuring the same concept. Divergent validity involves ensuring items across different factors have lower correlations (below 0.30) to confirm they measure distinct concepts. He explains that while a lack of divergent validity isn't always negative, it’s crucial to differentiate constructs, especially when assessing concepts like emotional exhaustion, depersonalization, and personal accomplishment within burnout studies.

Takeaways

• 🧐 The video discusses convergent and divergent validity, including their definitions, testing methods, and result interpretations.
• 🔍 To test for convergent validity, a correlation analysis is conducted, specifically a Pearson correlation on individual items rather than totals or averages.
• 📈 For convergent validity, the effect size of the correlation should ideally be between 0.30 and 0.70, indicating that items are measuring the same construct.
• 🟡 High correlation among items within the emotional exhaustion factor suggests they are measuring a similar concept.
• 🟢 The depersonalization items have slightly lower correlations but still fall within the acceptable range, indicating they measure a similar construct.
• 🔵 Personal accomplishment items have lower correlations, suggesting they may be interpreted or responded to differently by participants.
• 🤔 Divergent validity is tested by comparing items across different factors, aiming for an effect size below 0.30 to show they measure distinct constructs.
• 🚫 Lack of divergent validity (high correlations between different factors) doesn't necessarily indicate a problem, as some constructs are expected to be related.
• 😐 Face validity, although not always considered scientific, is important for understanding if variables make sense in relation to each other.
• 📝 The script provides an example of how to write up results, indicating whether convergent or divergent validity is present between different factors.
• 🔄 The process of reviewing each item for validity is repeated for all factors to ensure that they measure distinct constructs where appropriate.

Q & A

What is convergent validity?

Convergent validity refers to the degree to which two or more measures that theoretically should be related are indeed observed to be related.

What is divergent or discriminant validity?

Divergent or discriminant validity is the extent to which a test measures something different from what other tests are designed to measure.

What is the recommended effect size range for convergent validity?

The effect size of the correlation for convergent validity should be between 0.30 and 0.70.

What does it mean if an item's correlation is over 0.70 in terms of convergent validity?

While an item's correlation over 0.70 is not ideal, it is of minimal concern as long as most items meet the requirements, indicating they are measuring relatively the same thing.

What is the significance of conducting a correlation analysis on each item rather than the totals or averages of the factors?

Analyzing each item individually helps to ensure that each item is contributing to the measurement of the intended construct and not just the overall factor.

What are the three factors of burnout mentioned in the script?

The three factors of burnout mentioned are emotional exhaustion (EE), depersonalization (DP), and personal accomplishment.

Why is it important to look at the correlations between different factors when assessing divergent validity?

Examining the correlations between different factors helps to ensure that each factor is distinct and not simply a repetition of another, which would indicate divergent validity.

What does it imply if the effect size between two factors is below 0.30?

An effect size below 0.30 suggests that the two factors are not highly related, supporting the idea that they are measuring different constructs.

What does it mean if divergent validity is not revealed between two factors?

If divergent validity is not revealed, it means that the two factors are moderately to highly correlated, which could indicate that they are measuring similar or overlapping constructs.

Why is face validity considered important despite some researchers dismissing it as unscientific?

Face validity is important because it relates to the common-sense understanding of whether a measure appears to tap the construct it is supposed to measure, which can provide initial insight into the validity of a test.

How can you interpret the results of a correlation analysis for personal accomplishment in terms of divergent validity?

If the correlation between personal accomplishment and the other factors of burnout is low and not significant, it suggests that personal accomplishment is a distinct construct that is not highly related to emotional exhaustion or depersonalization.

Outlines

00:00

🔍 Understanding Convergent and Divergent Validity

In this paragraph, Dr. Dan introduces the concepts of convergent and divergent validity in the context of psychological assessment. He explains that convergent validity is demonstrated when items within a factor correlate with each other, typically with an effect size between 0.30 and 0.70. The example provided involves assessing burnout, with factors such as emotional exhaustion, depersonalization, and personal accomplishment. Dr. Dan discusses the results of a Pearson correlation analysis, noting that while one item exceeds the recommended effect size, it is not a significant concern. He emphasizes that high correlations within factors indicate that the items are measuring similar constructs. Divergent validity, on the other hand, is when items from different factors do not correlate highly, ideally with an effect size below 0.30. Dr. Dan points out that moderate to high correlations between emotional exhaustion and depersonalization are expected, as these constructs are often experienced together.

05:00

📊 Evaluating Discriminant and Face Validity

Dr. Dan continues the discussion by exploring discriminant validity, which is shown when items from different factors do not correlate highly, indicating that they measure distinct constructs. He uses the example of emotional exhaustion and personal accomplishment, where the correlations are low, suggesting that these factors are indeed separate. He also touches on face validity, a concept that some researchers may dismiss but which Dr. Dan argues is important. Face validity refers to whether a test makes sense to the average person and whether the items appear to measure what they are intended to measure. Dr. Dan suggests that while high correlations between certain factors may not indicate discriminant validity, they can be acceptable if they align with face validity and theoretical expectations. He concludes by reiterating the importance of understanding when to expect convergent and divergent validity in psychological assessments.

Keywords

💡Convergent Validity

Convergent validity refers to the degree to which two measures that are supposed to be related are actually related. In the video, it is demonstrated by how items measuring the same concept (such as emotional exhaustion) have high correlations between each other, typically falling between 0.30 and 0.70. This indicates that the items are effectively measuring the same construct.

💡Divergent Validity

Divergent validity, also known as discriminant validity, is the degree to which concepts or measurements that are not supposed to be related are actually distinct. In the video, this is tested by examining whether items measuring different constructs (like emotional exhaustion and personal accomplishment) show low correlations (below 0.30). If correlations are low, it suggests that these items are measuring different constructs, as expected.

💡Correlation Analysis

Correlation analysis is a statistical method used to evaluate the strength and direction of the relationship between two variables. In the video, Dr. Dan emphasizes that a Pearson correlation analysis is used to assess the relationships between individual items in different scales of burnout (emotional exhaustion, depersonalization, and personal accomplishment). This analysis helps to determine both convergent and divergent validity.

💡Effect Size

Effect size refers to the magnitude of the relationship between variables. In the context of the video, effect size is used to describe the strength of the correlations between different items on the burnout scales. For convergent validity, an effect size between 0.30 and 0.70 is considered ideal, while for divergent validity, an effect size below 0.30 is preferred.

💡Emotional Exhaustion

Emotional exhaustion is one of the factors used to measure burnout. It refers to feelings of being emotionally overextended and depleted of emotional resources. In the video, Dr. Dan explains that items measuring emotional exhaustion should have high correlations with each other to demonstrate convergent validity, indicating that they are capturing the same aspect of burnout.

💡Depersonalization

Depersonalization is another factor related to burnout, characterized by a detached or impersonal response towards recipients of one's care or service. In the video, depersonalization is compared with emotional exhaustion and personal accomplishment to assess divergent validity. Lower correlations with these other factors suggest that depersonalization is a distinct aspect of burnout.

💡Personal Accomplishment

Personal accomplishment refers to feelings of competence and successful achievement in one’s work. In the video, personal accomplishment is contrasted with emotional exhaustion and depersonalization. Dr. Dan points out that lower correlations between personal accomplishment and the other two factors support the idea that personal accomplishment is a separate construct from the negative experiences of burnout.

💡Pearson Correlation

Pearson correlation is a statistical measure that assesses the linear relationship between two variables. In this video, Dr. Dan uses Pearson correlations to evaluate the degree of association between different burnout items, which helps in testing both convergent and divergent validity. A Pearson correlation close to 1 indicates a strong relationship, while a correlation close to 0 suggests no relationship.

💡Burnout

Burnout is a psychological syndrome that arises from prolonged exposure to stress in the workplace, and it is characterized by emotional exhaustion, depersonalization, and reduced personal accomplishment. The video focuses on how burnout is measured through these three factors, using correlation analysis to test for both convergent and divergent validity of the items that assess burnout.

💡Face Validity

Face validity refers to the extent to which a test appears to measure what it claims to measure, based on a superficial assessment. Although Dr. Dan mentions that face validity is often dismissed as a non-scientific concept, he argues that it is still important in understanding whether variables logically relate to each other. For example, correlations between emotional exhaustion and depersonalization may make sense intuitively, even if they don't show perfect divergent validity.

Highlights

Convergent and Divergent validity are discussed in the context of validity testing.

Correlation analysis is the first step for testing convergent and divergent validity.

Pearson correlation is used to analyze each item rather than totals or averages of factors.

Three factors of burnout are identified: emotional exhaustion, depersonalization, and personal accomplishment.

Convergent validity requires correlation effect sizes between 0.30 and 0.70.

Items associated with emotional exhaustion mostly meet the convergent validity criteria.

One item exceeds the 0.70 threshold but is not a significant concern.

Depersonalization items have slightly lower but acceptable effect sizes.

Personal accomplishment items show lower effect sizes, indicating possible differences in participant responses.

Divergent validity is tested by comparing different factors with an effect size below 0.30.

Correlations between emotional exhaustion and depersonalization are above 0.30, indicating a lack of divergent validity.

Face validity is considered important despite some researchers' skepticism.

Divergent validity is revealed between emotional exhaustion and personal accomplishment with effect sizes well below 0.30.

The significance of divergent validity is discussed in terms of measuring distinct constructs.

The importance of understanding convergent and divergent validity within factor analysis is emphasized.

The practical implications of validity testing for assessing burnout are highlighted.

The video provides a comprehensive guide on how to interpret results of validity testing.

Transcripts

hello this is Dr Dan this video we will

discuss convergent and Divergent

validity including their definition

testing for convergent and Divergent

validity and interpreting the results

the first thing you'll need to do with

your data

is conduct a correlation analysis in

this case I've I've done a Pearson

correlation and you want to run a

correlation not on any totals or

averages of the factors but on each item

in this example over here on the left

you can see there are three items

associated with EE which is emotional

exhaustion three items associated with

DP which is depersonalization and three

items associated with personal

accomplishment

these are

three factors that are included in the

concept of burnout just for your

information

now when we want to look at convergent

validity

researchers will say that the effect

size of the correlation should be

between 0.30 and 0.70

and so what we have here in each of

these color-coded

areas here and yellow green and blue we

have the correlations with emotional

exhaustion here in yellow

depersonalization and green and personal

accomplishment in blue

as you can see with emotional exhaustion

each of these three items correlate with

an effect size that for the most part

meets the requirements now there is one

that's over 0.70 but that's that's

really of minimal concern what we want

to be concerned about here

is just that they are correlating with a

high effect size meaning that each of

these three items is measuring

relatively the same thing that's kind of

a sort of a basic way of understanding

that

the for depersonalization the the items

are a slightly lower however they still

fall within the parameters of 0.30 and

0.70 effect size

and then when we look at personal

accomplishment we see these items are

much or considerably lower they they do

fall within the parameters of 0.30.70

for the most part

however at a low effect and so basically

what this means is those three items

when participants take this assessment

uh they they may be answering them

slightly differently in other words uh

those items are phrased in such a way

where the participants are not going to

respond uh in like an identically for

each one of those three items less so or

more so depending how you want to look

at it for emotional exhaustion where

they would be if they're emotional

exhausted for the first item

um you know if they interpret that form

of exhaustion for that well they

probably will for the second and the

third as well because those are probably

phrased very similarly

okay so in this case again this is

convergent validity

where each of these

three factors within each of these three

they are converging on each other

another way of looking at this which I

discussed in another video is Chromebox

Alpha now that's looking at reliability

but in many ways the concepts are I

don't want to say necessarily the same

but extremely similar where we are

looking to see if the three items are

kind of matching up with each other

now let's look at

divergent or also sometimes called

discriminant validity in this case we do

not necessarily want to a high degree

the items correlating with each other so

in this case the effect size should be

below 0.30

what we have here is in yellow again

emotional exhaustion there's these these

three items and now we are going to

compare not within

emotional exhaustion itself not within

that factor but we are going to compare

that to the other two factors so in this

case we look at the correlations between

emotional exhaustion and

depersonalization

all three of the

ee factors and the BP factors and then

the same with PA personal accomplishment

in this case

we want to look again to show diversion

or discriminate validity the effect size

below 0.30 and in this case they are not

they're above 0.30 at uh moderate to

sometimes a high effect size now is this

a bad thing well it does not show

discriminant validity or Divergent

validity and I use those words again

interchangeably

however that's not necessarily a bad

thing because we also expect

that these items will correlate because

research will continuously show that

emotional exhaustion and

depersonalization If you experience one

you're likely experiencing the other as

well

now

many researchers will say there is no

such thing as face validity that it's

not really a scientific concept however

I'll demonstrate here that it's actually

extremely important in the in the

concept does apply to something like

this so in other words face validity

has to do with just kind of like Common

Sense understanding your variables and

so at face value if you will we would

expect somewhat of a correlation here so

therefore we would expect that the

effect size maybe will you know not

necessarily be under 0.30 now I would

say it'd be more problematic if these

were all like you know very high effect

like .70 and perhaps higher because then

maybe they're kind of measuring the same

thing

okay so this is acceptable but we're

looking here and how we would write this

up is basically say that diversion or

discriminant validity was uh you know

not revealed between emotional

exhaustion depersonalization now on the

other hand

it was between emotional exhaustion and

personal accomplishment because all of

these effect sizes here first off

they're not even significant except for

one and they are well below 0.30

now what does that tell us well

basically what that says is first off we

do have divergent or discriminant

validity and I know at again you know

face value because I understand the

measure that that seems logical because

just because someone is emotionally

exhausted in their job they'll likely

experience also depersonalization but

not necessarily personal accomplishment

because someone can be emotionally

exhausted in their job but still have a

great sense of accomplishment if they

really believe they're helping someone

out so that is why you do not

necessarily see a high correlation

between the two and you definitely see

discriminant validity that these are

measuring two different things

now when we compare depersonalization so

now what we're basically doing is we're

doing the exact same thing

that we did with emotional exhaust and

now we're looking at depersonalization

so in yellow you have depersonalization

items and we're looking now at emotional

exhaustion which is to the left in this

case and personal accomplishment which

is below

just like we you know just like we

experienced below excuse me previously

and we we again have these these values

over here which we already discussed uh

in this case new values that we're now

looking at is depersonalization personal

accomplishment which here again low

values revealing that these are are you

know demonstrate that these are two

different

Concepts that we're looking at two

different constructs if you will and uh

and the reasoning is the same as I gave

previously with emotional exhaustion and

deep and and personal accomplishment is

depersonalization and personal

accomplishment can be two different

things you can still experience

depersonalization but uh and be exhaust

and and experience burnout but also have

a sense of personal accomplishment and

then finally

the way you would look at personal

accomplishment is you look at all of

these items to the left which we've

already gone over in in the previous

ones I just wanted to show you these

three examples of how you review each

one

I hope that helps you out and again it

is important to understand that

convergent validity is something that we

perhaps want to see within each of our

factors Divergent validity is something

that we do not necessarily want to see

at high levels otherwise we're perhaps

measuring the same thing