Item Analysis - Difficulty Index and Discrimination Index || Assessment and Evaluation || tsin-eng

Summary

This video covers item analysis, a statistical technique for selecting and rejecting test items based on their difficulty and discriminating power. It outlines how to calculate item difficulty, discrimination, and point-biserial correlation to ensure quality test construction. The video emphasizes the importance of analyzing items to detect learning difficulties, refine instructional materials, and improve future test development. It also highlights the limitations, such as the inability to analyze essay items and the need for caution when deleting or revising test items based on the analysis. Practical steps and formulas are provided to help educators evaluate test items effectively.

Takeaways

• 😀 Item analysis is a statistical technique used to evaluate and select or reject test items based on their difficulty and discrimination properties.
• 😀 The primary purpose of item analysis is to select the best items for a test, identify content defects, detect class learning difficulties, and improve test construction skills.
• 😀 Item analysis provides data on three key components: item difficulty (P-value), item discrimination (D-value), and destructor analysis (how well alternatives function).
• 😀 Item difficulty (P-value) measures how easy or hard an item is, with values ranging from 0 to 1. A higher value indicates an easier item, while a lower value indicates a more difficult one.
• 😀 If the P-value is above 0.85, the item is considered too easy, and if it's below 0.20, the item is considered very difficult and needs revision.
• 😀 The formula for calculating item difficulty (P) is: P = (Ru + Rl) / T, where Ru and Rl are the correct answers from the upper and lower groups, respectively, and T is the total number of test-takers.
• 😀 Item discrimination (D-value) indicates how well an item distinguishes between high and low scorers. A higher D-value means better discrimination.
• 😀 The formula for calculating item discrimination (D) is: D = (Ru - Rl) / (T / 2), where Ru and Rl represent correct answers from the upper and lower groups, respectively.
• 😀 A high D-value (closer to 1) indicates a good item, while a D-value near 0 or negative suggests the item is ineffective and should be removed.
• 😀 Point-biserial correlation measures the relationship between an item score and total test score, with a negative correlation indicating a problematic item that should be rewritten.
• 😀 Sensitivity to instructional effects measures how well instructional material improves test performance, with values close to 1 indicating effective teaching material and values of 0 or negative showing no effect.

Q & A

What is item analysis in the context of educational testing?

Item analysis is a statistical technique used to select and reject test items based on their difficulty value and discriminating power. It helps in improving the quality of test items by assessing their effectiveness in measuring the intended learning outcomes.

What are the main purposes of item analysis?

The main purposes of item analysis are to select the best items for a test, identify structural or content defects in the items, detect learning difficulties, identify areas of weaknesses in students, enhance instructor skill in test construction, and identify areas of a course that need greater emphasis or clarity.

What are the three aspects of item analysis information?

Item analysis provides information about item difficulty, item discrimination, and destructor analysis. These help evaluate whether an item was too easy or difficult, how well it discriminated between high and low scorers, and whether all alternatives functioned as intended.

How is item difficulty calculated?

Item difficulty is calculated as the percentage of upper and lower 27% of students who answered the item correctly, after ordering their obtained scores in descending order. It is represented as a proportion between 0.0 and 1.00, known as the p-value.

What does a high p-value (above 0.85) indicate in item difficulty analysis?

A high p-value (above 0.85) indicates that the item is very easy, meaning almost all students got it correct. Such items should not be reused for future tests, as they do not effectively assess students' knowledge.

What does a p-value below 0.20 indicate?

A p-value below 0.20 indicates that the item is very difficult and should be reviewed for confusing language. Items with this low difficulty should be highlighted for potential re-instruction and should be removed from future tests.

How is item discrimination calculated?

Item discrimination is calculated by subtracting the number of correct responses in the lower group (rl) from the number in the upper group (ru), and then dividing by half the total number of students (t). The formula is: DI = (ru - rl) / (t / 2).

What does a low discrimination index (near or less than 0) signify?

A low discrimination index indicates that the item did not effectively differentiate between high and low scorers. This could mean the item was confusing for higher-scoring students, and it should be reviewed or removed from the test.

What is the purpose of point-by-serial correlation in item analysis?

Point-by-serial correlation is used to correlate item scores with the overall test score. It helps assess whether a student who performed well on the test missed a particular item, which can indicate the quality of that item.

What does sensitivity to instructional effects measure in item analysis?

Sensitivity to instructional effects measures how well an item reflects the effectiveness of instruction. It compares student responses before and after instruction to see if the item became easier for students post-instruction. Items with a higher positive value indicate a stronger instructional effect.