Psikometri 17_Reliabilitas bagian 4

Summary

This video explores the sources of measurement error, focusing on scoring differences and time sampling errors. It highlights how subjective scorer differences can affect assessment results, even with guidelines in place, and introduces techniques like inter-rater reliability and Pearson correlation to assess reliability. It then discusses time sampling errors, emphasizing the impact of test timing on results, with examples like verbal comprehension vs. attention capacity. The video concludes by questioning the ideal time gap for retesting in rapidly changing conditions like exam anxiety, noting that test-retest methods may not be suitable for such variables.

Takeaways

• 😀 Scoring differences, even with assessment guidelines, can occur due to subjective considerations of the scorer, not due to carelessness.
• 😀 Using a scorer relay technique requires at least two independent assessors to reduce subjective errors and assess reliability.
• 😀 Pearson's product-moment correlation can be used to calculate the reliability of assessments. A correlation of 0.9 or higher means a 10% or less error rate.
• 😀 Time sampling error occurs when the wrong test time is chosen, impacting the evaluation of behaviors or traits that change over time.
• 😀 Certain abilities, such as verbal comprehension, can change faster than others like personality, affecting test reliability over time.
• 😀 The test-retest method can help assess reliability by repeating the test at different time intervals to the same person.
• 😀 There is no universal 'best' time interval for retesting as the interval should depend on the specific variable being measured.
• 😀 Short time intervals between tests may lead to memory biases, while long intervals can introduce factors like age and maturity changes.
• 😀 For rapidly changing traits like anxiety, a test-retest method may not be suitable, as the trait can shift too quickly.
• 😀 The timing of when anxiety peaks (e.g., before or minutes before an exam) varies for individuals, influencing the test design and retesting considerations.
• 😀 A retest of anxiety after the exam would likely yield different results compared to before, due to the rapid nature of the trait's changes.

Q & A

What is the first source of measurement error discussed in the script?

The first source of measurement error discussed is the difference between assessors, known as scorer differences. This occurs when subjective considerations by the scorer affect the assessment, even though guidelines are in place.

How can scorer differences impact the assessment results?

Scorer differences can lead to variations in assessments due to the personal interpretations or biases of the scorer, even when standardized assessment guidelines are used.

What method is used to calculate the reliability of assessment scores from different assessors?

The reliability is calculated by finding the correlation between the scores given by two or more independent assessors. This can be done using the Pearson product-moment correlation technique.

What correlation value indicates a reliable assessment according to the script?

A correlation value of 0.9 or higher indicates high reliability, meaning the error rate is 10% or less.

What is the second source of measurement error mentioned in the script?

The second source of measurement error is time sampling error, which occurs due to the wrong choice of test timing. This can affect the reliability of the test results, especially when the characteristic being measured changes over time.

How can time sampling errors be measured and minimized?

Time sampling errors can be minimized using the test-retest reliability technique, where a test is administered at different time intervals. The correlation between the results of the first and second tests is used to measure reliability.

Why is there no universally ideal time interval between tests?

There is no universal ideal time interval because factors like memory recall, aging, or maturity can affect test scores. The time interval must be appropriate for the specific trait being measured.

What are the challenges in selecting the appropriate time interval between tests?

If the time interval is too short, participants may remember their initial responses, affecting the results. If the interval is too long, other factors like aging or changes in the individual’s abilities may interfere with the test outcomes.

Why is the test-retest method not suitable for measuring traits like exam anxiety?

The test-retest method is not suitable for measuring traits like exam anxiety because such traits can change rapidly, and the time gap between tests could result in significant differences in anxiety levels, making the results unreliable.

What is the speaker's intention for future discussions as mentioned in the script?

The speaker intends to discuss other types of measurement errors in future sessions that may be more appropriate for assessing traits that change quickly, like anxiety.