LIBRENG SIT IN PSYCHOMETRICS EP. 12: VALIDITY OF PSYCHOLOGICAL TESTS (PT. 2)

Summary

This script delves into the concept of validity in psychological testing, emphasizing its importance in predicting future behaviors and thoughts. It distinguishes between concurrent and predictive validity, explaining how tests like the High Risk of Suicide Test (HRST) can forecast outcomes with high scorers. The speaker outlines steps for establishing validity, using criteria such as GPA or supervisor ratings, and highlights the benefits of predictive validity in hiring and preventive actions, ultimately advocating for its establishment in test development for better decision-making.

Takeaways

• 🔍 Validity in psychological testing refers to the ability of a test to measure what it claims to measure and to predict future thinking and behavior.
• 📊 There are two main types of validity discussed: concurrent validity and predictive validity. Concurrent validity assesses if a test can predict an already existing criterion, while predictive validity checks if it can predict a future criterion.
• 📚 The script explains that a criterion is a numerical expression or observable indicator of a test construct, such as GPA for intelligence or supervisor ratings for job efficiency.
• 📝 The importance of establishing validity is emphasized, as it helps in minimizing errors in hiring and making informed decisions, such as awarding scholarships or identifying students at risk of depression.
• 🔑 The concept of 'construct' is distinguished from 'criterion', with the former being an abstract concept measured by psychological tests and the latter being its concrete or numerical expression.
• 📉 The process of establishing validity involves administering a test, selecting an appropriate criterion, and then correlating the test scores with the criterion to determine the strength of the relationship.
• 📈 A high correlation coefficient, such as 0.93, indicates a strong relationship between test scores and the criterion, suggesting good predictive or concurrent validity.
• 📊 The difference between concurrent and predictive validity is highlighted, with predictive validity requiring a wait for the criterion to manifest, making concurrent validity potentially more time-efficient.
• 🛠️ The script provides a hypothetical example of a suicide risk test (HRST) to illustrate how predictive validity can be assessed and its importance in identifying individuals at risk.
• 🏫 An example scenario involving a school's chief psychometrician is used to explain the steps to determine if a test can predict students' future college grades, emphasizing the role of GPA as a criterion.
• 🏢 Another example involving a job efficiency test is given to demonstrate how a test's scores can be correlated with supervisors' ratings to establish concurrent validity in a workplace setting.

Q & A

What is the primary purpose of validity in psychological testing?

The primary purpose of validity in psychological testing is to ensure that the test measures what it claims to measure and can predict future thinking and behavior.

What are the two types of validity discussed in the script?

The two types of validity discussed are concurrent validity and predictive validity.

How does predictive validity differ from concurrent validity?

Predictive validity involves predicting future outcomes based on test scores, while concurrent validity involves correlating test scores with existing criteria.

What is a criterion in the context of psychological testing?

A criterion is a numerical expression or observable indicator of a test construct, such as school GPA for intelligence or supervisors' ratings for job efficiency.

Why is it important to establish the validity of a psychological test?

Establishing the validity of a psychological test is important to ensure it can accurately predict future thinking and behavior, which aids in decision-making processes such as hiring or scholarship allocation.

What is the significance of a high correlation coefficient in establishing validity?

A high correlation coefficient indicates a strong relationship between test scores and the criterion, suggesting that the test has good predictive or concurrent validity.

How can predictive validity be used in a corporate setting?

Predictive validity can minimize errors in hiring by identifying high and low performers based on test scores, leading to more informed hiring decisions.

What is the hypothetical test mentioned in the script for the purpose of explanation?

The hypothetical test mentioned is a 'High Risk of Suicide Test' (HRST), used to illustrate the concept of predictive validity.

What steps are involved in establishing concurrent validity for a test measuring intelligence in a school setting?

The steps include administering the test to incoming first-year students, waiting until their fourth year, and then correlating their test scores with their GPA to establish concurrent validity.

How does the case of the job efficiency test illustrate the concept of concurrent validity?

The job efficiency test is administered to existing employees, and their test scores are correlated with their supervisors' ratings from the previous month, demonstrating the test's ability to predict an existing criterion.

Why might a test developer prefer to establish predictive validity over concurrent validity?

A test developer might prefer predictive validity because it can predict future outcomes, which is more useful for long-term decision-making and planning compared to concurrent validity.

Outlines

00:00

🧠 Understanding Test Validity

The paragraph discusses the concept of validity in psychological testing, emphasizing that validity is not only about a test's ability to measure what it claims but also its capacity to predict future behavior and thinking. It introduces two types of validity: content validity and criterion-related validity. The latter is further divided into concurrent and predictive validity. Concurrent validity examines the test's ability to predict existing criteria, while predictive validity looks at its ability to forecast future criteria. The paragraph uses the example of a hypothetical suicide risk test to illustrate predictive validity.

05:03

📚 Criterion and Construct in Validity Testing

This paragraph delves into the difference between a 'construct' and a 'criterion' in psychological testing. A construct is an abstract concept like intelligence or personality, while a criterion is a concrete or numerical expression of that construct, such as GPA for intelligence. The paragraph explains that to establish concurrent or predictive validity, one must first identify an appropriate criterion. It also presents a case scenario where a psychometrician needs to determine if an entrance exam can predict college performance, suggesting steps such as administering the test to first-year students and correlating their scores with their fourth-year GPA.

10:17

🔢 Correlation Between Test Scores and GPA

The paragraph outlines a step-by-step process for assessing the predictive validity of a test. It describes administering an entrance exam to first-year students and later correlating their raw scores with their fourth-year GPA. The strength of the correlation coefficient indicates the test's predictive power. A high correlation suggests the test is effective in predicting academic performance, while a low correlation indicates the test's inability to predict future performance. This process is crucial for validating the test's predictive validity.

15:20

🏢 Applying Predictive Validity in Job Efficiency

This paragraph extends the concept of predictive validity to a job efficiency test. It describes a scenario where employees take a job efficiency test, and their scores are later correlated with their supervisors' ratings from the previous month. A high correlation coefficient indicates that the test can accurately predict job efficiency as perceived by supervisors. The paragraph highlights the importance of selecting an appropriate criterion that already exists to establish concurrent validity, as opposed to predictive validity where the criterion is yet to be determined.

20:36

📈 The Importance of Predictive Validity

The final paragraph underscores the significance of predictive validity in minimizing errors in hiring and taking preventive actions. It suggests that tests with high predictive validity can be used to identify high and low performers, thus aiding in decision-making processes such as hiring and scholarship allocation. The paragraph concludes by emphasizing the importance of establishing a test's concurrent or predictive validity to ensure its reliability in predicting future thinking and behavior.

25:36

📝 Conclusion and Future Discussion

The concluding paragraph invites viewers to engage with the content by asking questions or leaving comments. It also sets the stage for future discussions on additional categories of procedures for establishing test validity. The speaker expresses gratitude for the audience's attention and well-wishes, promising more insights in upcoming videos.

Keywords

💡Validity

Validity in the context of the video refers to the extent to which a psychological test measures what it is supposed to measure and can predict future thinking and behavior. It is a core concept that underpins the reliability and usefulness of tests. For example, the video discusses how a test's validity can be expressed through its ability to predict future actions, such as suicide risk or job performance.

💡Criterion

Criterion is defined as the numerical expression or observable indicators of a test construct. It serves as a standard for evaluating the construct being measured, such as intelligence or job efficiency. In the script, criterion is used to establish concurrent or predictive validity by correlating test scores with concrete outcomes like GPA or supervisor ratings.

💡Construct

A construct in psychological testing is an abstract concept that the test aims to measure, such as intelligence, personality, or stress. The video explains the difference between a construct and a criterion, emphasizing that while a construct is abstract, a criterion provides a concrete measure of that construct.

💡Predictive Validity

Predictive validity is the ability of a test to predict future outcomes based on current measurements. The video script uses the example of a high-risk suicide test (Hrst) to illustrate how predictive validity can indicate the likelihood of future behavior, such as the chance of an individual committing suicide.

💡Concurrent Validity

Concurrent validity is the ability of a test to predict an existing criterion at the time of testing. The script contrasts concurrent validity with predictive validity, explaining that concurrent validity involves correlating test results with existing measures, such as using supervisor ratings to validate job efficiency tests.

💡Criterion-Related Validity

Criterion-related validity encompasses both concurrent and predictive validity and is the degree to which a test can predict external criteria. The video discusses how establishing this type of validity is crucial for tests that aim to forecast future thinking or behavior.

💡High-Risk Suicide Test (Hrst)

The High-Risk Suicide Test (Hrst) mentioned in the script is a hypothetical example of a psychological test designed to predict the likelihood of an individual committing suicide. It serves to illustrate the concept of predictive validity in action.

💡Correlation

Correlation in the video refers to the statistical method used to measure the relationship between two variables, such as test scores and GPA or supervisor ratings. It is used to determine the strength and direction of the relationship, which can indicate the validity of a test.

💡Psychometrician

A psychometrician is a professional who specializes in the measurement of psychological variables. In the script, the role of the school's chief psychometrician is highlighted in the context of evaluating the validity of a test designed to predict college grades.

💡Preventive Actions

Preventive actions in the video script refer to the steps taken based on test results to prevent or mitigate potential issues, such as offering special programs for students identified as being at risk of depression through psychological testing.

💡Decision Making

Decision making is a process that can be informed and facilitated by the predictive validity of tests. The video emphasizes how tests that can forecast future performance or behavior can assist in making decisions such as hiring, scholarships, or interventions.

Highlights

Validity in psychological testing is the ability of a test to measure what it claims to measure.

Validity also encompasses the test's ability to predict future thinking and behavior, and to be taken seriously by test-takers.

Criterion prediction procedures are used to establish validity, including concurrent and predictive validity.

Concurrent validity asks if a test can predict future thinking or behavior, while predictive validity looks at existing criteria.

A criterion is a numerical expression or observable indicator of a test construct, such as GPA for intelligence.

Constructs are abstract concepts like intelligence, while criteria provide concrete measures like school grades.

Establishing concurrent or predictive validity involves correlating test scores with criteria like GPA or supervisor ratings.

A high correlation coefficient indicates strong predictive validity, suggesting the test can accurately predict future performance.

Low predictive validity implies a weak relationship between test scores and future outcomes.

Predictive validity is crucial for minimizing errors in hiring and for taking preventive actions in education and mental health.

A test with high predictive validity can identify high and low performers, aiding in decision-making for scholarships and hiring.

Establishing validity is essential for developing tests that can predict future thinking patterns and behaviors.

The importance of predictive validity in making decisions for scholarships and hiring based on test performance.

The process of establishing validity through concurrent and predictive validity procedures is vital for test development.

A comparison between tests with high concurrent validity versus high predictive validity, emphasizing the preference for predictive validity.

The practical applications of tests with established validity in the corporate world and educational settings.

The role of predictive validity in identifying students at risk of depression for preventive mental health programs.

The upcoming discussion on additional categories of procedures for establishing test validity in the next episode.

An invitation for questions and comments from the audience, emphasizing the presenter's commitment to answering them.

Transcripts

hello good day sir over here

it is on the validity of psychological

tests

this is the second episode that we have

talking about established

tests remember validity is the ability

of the test to measure

what it claims to measure but we

discussed in the first

video the one before this namarami

expressions other than

the ability of the test to measure what

it claims to measure

the ability of the test to predict

future thinking and behavior

is also validity the ability of the test

to make people

take the test seriously is also an

expression

of validity

is the second category of validity

procedures

the criterion prediction procedures

specifically

established validity gamete and

concurrent

attack predictive validity now what is

different

in the second procedures compared to the

first one

remember

on the screen these categories

ask different questions about validity

the un's first box was nothing known

last episode is your test measuring

the construct it is supposed to measure

this box is very different number two

it asks the question can a test predict

future thinking

and behavior the key word

of thinking or behavior in the future

so if that is your question

that your test can do then

a procedures to prove that

concurrent validity or predictive

validity

remember another definition

or another expression of validity is the

ability of the test to predict

future thinking and behavior

in the future that in itself is an

expression

of the validity of test

hypothetical test

test

for the sake of explanation let's say we

have a test

whole words risk of suicide test or the

hrst

you look at the manual of this test

and predictive validity

what does that mean if a certain

psychological test

has high predictive

who will be taking this test for example

let's pick up the high scorers

more or less more than 50 chance they

will commit suicide

on the other hand the the reverse is

also true

so we can easily predict what will

happen to

test takers who are high scorers versus

who are low scorers but

why are we so confident to predict what

will happen

to high scorers and low scorers because

this test

has good predictive validity

thinking or behavior now how do i know

if my test has that ability

but let's first define the word

criterion

going back to that box

procedures

so what do you mean by criterion if it's

a london criterion

this is the numerical expression or the

observable indicators

of a test construct remember in the

last video we discussed the meaning of

the word

construct construct means at the young

men

measuring psychological tests it could

be

intelligence or personality or stress

of intelligence

numerical expressions of intelligence

examples would be school gpa in school

gpa

young grades escalahan can be the

numerical expression of intelligence

or young graduates is an iq test that is

also

a a criterion of intelligence

the number of books that the person has

or the number of books that the person

has already

read so again criterion is the concrete

or the numerical expression

of the construct another example

extroversion

and no i'm possibly numerical expression

and extra version

when you see this number foreign

the number of of social gatherings that

a person attends when they're in how

about aggression

aggression is the construct

is a psychological test but what could

be a possible criterion

of that construct

can reflect how aggressive a person is

with the number of school offenses

so that is the difference between

construct and criterion

construct is abstract while criterion is

numerical

it's very important criterion

in establishing concurrent or predictive

validity now let's move on to case

number one parapakusa panang

you read the case on your own pause the

video if you need to

and then we'll discuss

so based on case number one it is the

school's chief psychometrician what

would you do to find

out whether this test can really predict

students who will get

high college grades

so let's talk about the steps to solve

this problem

step one shamprek

administered incoming first-year

students

again what is criterion the numerical

expression of a construct

definitely intelligence so given that

your test is measuring intelligence

for intelligence in the context of this

case

criterion that we are going to assign

their gpa

on their fourth year step four in

administering municipalities

so what you do is you accept this

incoming first-year students

you accept these people in your school

and then anti-inflammatory syllabi

okay now you are ready to go or to do

step 5

scores remember scores that we are going

to correlate here

una scoring

what is their raw score in that exam

you are going to correlate correlate

that with their gpa

right now that they are now already in

their fourth year

so if you can picture if you can follow

my line of thought here

entrance exam result

let's say we already did that

correlation coefficient

there is a direct and strong

relationship between

their entrance exam test score and their

fourth year

gpa so nothing direct and strong

positive point 93 what does that mean

it means

those students who scored high in the

entrance exam also had high

gpa while those students who scored

low in entrance exams

gpa no fourth year to make the long

story

short if your test is performing like

this

to predict who will be the good students

and who will be the not so good students

their entrance exam three years ago

predicted their scores

predicted their gpa when they were

already

in their fourth year so it literally mo

entrance examine young

so it is a good test it is a test that

can predict

future performance

positive point 18 meaning there is a

weak relationship

between the two

relationships

whether you are high or low whether high

or low on entrance example has nothing

to do

with your fourth-year gpa

entrance

gpa so your test cannot predict future

performance

you know nothing procedure nyan is

called predictive validity

where in the process we test the ability

of the test

to uh predict a future criterion

in the future now and this advantage and

predictive validity

is reflected in this case

okay so let's talk about this case read

on your own silently

you job efficiency test so you

administer the job efficiency test

to the existing employees and then

punish cornell remember you

construct test data is job efficiency

step two this time we select criterion

criterion that we are going to select is

a criterion

that is already existing remember in

predictive validity

young criterion hidden

after three years in the previous case

but

given my validity

so what do we do we select a criterion

that is already existing

given our test construct the job

efficiency

why did nothing go meet the new

supervisors rating that these

employees had last month

so that will be the criterion that's a

good criterion

for job efficiency

supervisors rating is a numerical

expression

of job efficiency supervisors rating

existing nash

step 3 now you are going to correlate

the two scores

jung scored nanaku hanila from the job

efficiency test

against the supervisors rating nanaku

hanila

from last month so fast forward let's

say kenora

their job efficiency score second column

you supervise us rating in the last

month

correlation coefficient is positive 0.93

those employees who scored high in job

efficiency tests

had high supervisors rating last month

while those employees who scored low in

job efficiency tests

happens to be or happens to get low

supervisors rating last month

young job efficiency tests

has the ability to predict supervisors

rating

last month okay

validity in this in this procedure

we are testing the ability of a test to

predict

an already existing criterion

exists

in criterion in predictive validity the

criterion is not yet existing

okay so definitely between the two

a must time efficient young concurrent

validity

because in predictive validity you still

need to wait

for the criterion to exist

in terms of time compared to predictive

validity

because it has higher predictive

validity than

the than test b

validity so let's be more specific here

for the sake of illustration

predictive validity now let's pick

the 10 high scorers 10 high scorers

hrst now concurrent validity and

establish

versus 10 high

compared to a test with only a high

concurrent

validity dynamics a job efficiency test

let's say job efficiency tests

concurrent validity versus same test

job efficiency test mata and predictive

validity

let's speak 10 high scorers for each

test

predictive validity 10 high scores

concurrent validity so that's the

difference between a test

with high predictive validity and a test

with high concurrent validity

is a test development must be preferred

test developers to establish predictive

validity of the test

more than concurrent validity

establishment validity

non-future thinking patterns or

behaviors

now before we close this short video on

this second

category of establishing test validity

i want you to realize how important or

how useful

pets are of ability to predict

the future number one

predictive validity it can minimize

errors in hiring especially in the

industrial world or in the corporate

world

foreign form in your hiring

because you can trust your test your

test is a good predictive validity

performers

performer

that can really predict sino and high

performer

sin and low performer mas malition

chance

na you will hire someone

in your company number two this test can

also be used for preventive actions

that can predict who are the students

more likely to be depressed

mata's a tendency to be depressed

imagining more he administers

at the start of the semester

and you care about the mental health of

your students

who are high scorers in that test

special program dealing with depression

depressed yes individual

based on the test that can predict who

will be depressed

a test that can predict future thinking

or behavior is very useful

for decision making

scholarships that can predict

who will perform well in a school who

will not

it makes decision making easier

young scholarship

[Music]

but since the test tells me that they

will be high performers in the future

scholarship

so it's also very important that if you

develop a test

future thinking and behavior and how do

you do that by establishing the

concurrent

or the predictive validity of the test

from our short discussion of these

procedures in establishing the validity

of the test

so next episode not in pagosa

categories of procedures on how to

establish the validity

of the test if you have any questions

feel free to message me or to post a

comment down below

and i'm going to do my best to answer

your questions thank you everyone for

listening

as always god bless