Item Analysis

Summary

In this lesson, Teacher Claire explains the intricacies of item analysis in assessments, focusing on determining item difficulty and discrimination indices. She outlines essential steps, including ranking student scores and analyzing upper and lower performance groups. The session covers the formulas for calculating difficulty and discrimination, illustrating how to assess the effectiveness of test items. Claire emphasizes the importance of maintaining a desired level of difficulty and the role of distractors in ensuring a valid test. The lesson concludes with exercises for students to reinforce their understanding of the concepts discussed.

Takeaways

• 😀 Item analysis is a crucial process for evaluating student responses to test items, helping to identify which questions are effective.
• 📊 The process involves ranking student scores and categorizing them into upper and lower performance groups for comparison.
• 🔍 The three main criteria for assessing test items are item difficulty, discrimination index, and the effectiveness of answer alternatives.
• 📉 Item difficulty, or difficulty index, measures how many students answered a question correctly and should ideally be above 0.8.
• 🤔 A positive discrimination index indicates that a test item can effectively differentiate between high and low-performing students.
• 🚫 Items with a negative discrimination index suggest poor validity and should be revised or rejected.
• ✅ Good distractors are essential for effective test items; they should attract lower-performing students while being less appealing to high achievers.
• 🧮 The formulas for calculating difficulty and discrimination indices are straightforward, aiding in the quantitative assessment of test items.
• 📚 Teacher Claire provides exercises to enhance understanding of item analysis, emphasizing practical application.
• 🔗 Resources for further study and downloadable solutions are available through a link in the video description.

Q & A

What is item analysis?

Item analysis is the process of investigating student responses to each test item to identify items with desirable characteristics that can be retained and those with undesirable characteristics that can be revised or rejected.

What are the preliminary steps in conducting item analysis?

The preliminary steps include ranking students' scores from highest to lowest, identifying the top 27% and bottom 27% performing students, setting aside the middle 46%, and tabulating responses from the upper and lower groups.

How is the index of difficulty calculated?

The index of difficulty is calculated using the formula DI = R / N, where R is the total number of students who answered correctly, and N is the total number of students who took the test.

What is considered a desirable index of difficulty?

A desirable index of difficulty should not be lower than 0.8, indicating that the item is not too easy or too difficult.

What does the discrimination index measure?

The discrimination index measures the difference in the proportion of correct responses between high-performing and low-performing students, indicating how well the item differentiates between these groups.

What is the formula for calculating the discrimination index?

The discrimination index is calculated using the formula DS = (PU - PL) / N, where PU is the proportion of correct responses from the upper group, PL is the proportion from the lower group, and N is the total number of test takers.

What indicates a positive discrimination index?

A positive discrimination index indicates that high-performing students answered the item correctly more frequently than low-performing students, suggesting that the item is effective.

What happens if an item has a negative discrimination index?

A negative discrimination index suggests that low-performing students answered the item correctly more often than high-performing students, indicating poor item validity and potential ambiguity, which may lead to rejecting the item.

What is effectiveness of alternatives in item analysis?

Effectiveness of alternatives, or destructor analysis, evaluates the incorrect options (distractors) to determine if they effectively attract responses from lower-performing students, thus indicating the item's quality.

How does one determine if an item is good based on destructor analysis?

An item is considered good if its distractors attract more responses from lower-performing students compared to those from higher-performing students; otherwise, it may be seen as ambiguous.