Standardized Testing

Summary

In this video, Dan Hickey explores standardized testing, discussing its practices, principles, and policies. He explains what makes a test standardized, including the use of a common item bank and standardized scoring. Hickey touches on the role of educational and psychological standards, the application of item response theory, and the evolution of testing formats with technology. He also addresses the interpretation of test scores, the controversy surrounding their use, and the challenges in leveraging them for instruction improvement. The video concludes with a call for educators to understand standardized testing's basics and its impact on their practice.

Takeaways

• 📚 Standardized tests involve all test takers answering the same questions, often drawn from large item banks with known difficulty.
• 🎯 Standardized tests are scored in a consistent manner, though they may not always be aligned with educational or psychological standards.
• 🔬 Many standardized tests, especially those using psychological measurement, are aligned to theoretical constructs rather than formal educational standards.
• 🧠 Item Response Theory (IRT) allows test developers to understand the relative difficulty of test items, making it possible to equate different versions of the same test.
• 📝 Standardized tests often involve selected-response items, but performance assessments and writing tests are notable exceptions.
• 💡 New developments in standardized testing, such as evidence-centered design and adaptive tests, are changing how tests are structured and administered.
• 🏫 Standardized tests are often dictated by the educational context, meaning educators may have limited control over their use.
• ⏳ Standardized testing consumes significant time and resources, affecting instructional time and costs in schools.
• 📈 Interpreting standardized test scores can be done using percentiles, grade-level equivalents, or more complex scale scores, which are difficult to understand but more accurate.
• 🎓 Achievement test scores are used for evaluating students, teachers, and schools, but their effectiveness in improving instruction is highly debated.

Q & A

What is a key difference between standardized tests and other types of assessments?

One key difference is that standardized tests require all test takers to answer the same questions, often drawn from a large pool of carefully constructed items with known difficulty.

What is item response theory (IRT) and why is it important in standardized testing?

Item Response Theory (IRT) is a psychometric technique that helps test developers determine the relative difficulty of each item in a test. This allows for the creation of different versions of a test and enables the equating of scores across these versions.

How do standardized writing tests differ from traditional multiple-choice standardized tests?

Standardized writing tests require test takers to respond to the same prompt, and their responses are scored in a standardized manner, often by both computers and humans, unlike multiple-choice tests which involve selected responses.

What is evidence-centered design and how is it impacting standardized tests?

Evidence-centered design is a newer approach in standardized testing that allows test takers to answer different items based on their responses to previous questions. This model, used by organizations like the Smarter Balance Assessment Coalition, is changing the traditional formats of standardized tests.

What challenges come with using standardized performance assessments?

Standardized performance assessments face difficulties because they are more open-ended, making it harder to apply the psychometric assumptions and techniques that work well with selected-response items.

What is the difference between the mean and median when interpreting test scores?

The median represents the middle score in a distribution, while the mean is the average score. The median is easier to understand but the mean provides a more accurate picture of central tendency.

How is the standard deviation used in interpreting test scores?

Standard deviation measures the spread of scores around the mean. In a normal distribution, approximately 68% of scores fall within one standard deviation above or below the mean, while 95% fall within two standard deviations.

Why have percentile scores and grade-level equivalents been largely replaced by scale scores?

Percentile scores and grade-level equivalents are easy to interpret but can be misleading depending on the norming group. Scale scores, while more complex, provide more accurate comparisons across different groups and test versions.

Why is it problematic to use achievement test scores to compare schools?

Comparing schools based on achievement test scores has often led to negative consequences, such as overemphasis on test preparation rather than instruction improvement, and the use of scores for political purposes rather than meaningful educational reform.

What is a common criticism of using standardized tests to evaluate teachers, as seen in initiatives like Race to the Top?

A common criticism is that using standardized test scores to evaluate teachers is problematic because these scores do not directly reflect the quality of instruction. The tests are often disconnected from classroom activities, making it difficult to link test scores to teaching effectiveness.