SwaGrader -- evaluation of the students, by the students, for the MOOCs

Summary

This presentation introduces Sol Cratera, a peer grading tool designed to address common issues in large class evaluations. It utilizes the Tropican mechanism to discourage strategic behaviors like grade inflation or reduction, ensuring fair assessments. Developed by researchers from IIT Kanpur and UC Davis, the tool allows for class creation, student enrollment, and assignment management. It features a grading method where students can be both graders and gradees, with the ability to upload and split assignments. Tropican uses 'probes' for quality control, and the tool provides progress tracking and regrading requests. The presentation concludes with a demonstration of the final grade distribution.

Takeaways

• 📚 Peer grading is a method where students evaluate each other's work.
• 🔍 Traditional peer grading tools may overlook potential issues like strategic behavior by graders.
• 🌟 The tool 'Sol Cratera' introduces a mechanism called 'Tropicana' to discourage strategic behaviors in peer grading.
• 📈 Tropicana ensures that it's in the best interest of peer graders to grade honestly.
• 👥 Developed by researchers from IIT Kanpur and UC Davis, the tool aims to improve the peer grading process.
• 🚀 The tool allows for user roles such as students, teaching assistants, and instructors, each with different permissions.
• 📝 Assignments can be created and uploaded with answer keys for graders to reference.
• 📑 The tool automatically splits bulk assignments into individual submissions for grading.
• 🔍 Tropicana uses 'probes' to monitor the quality of peer grading, with instructor-graded assignments as the gold standard.
• 📊 Instructors can monitor the progress of the peer grading process and view individual grader performance.
• 🔒 Once completed, grades are frozen, and a final Tropicana score is provided for the peer grading exercise.

Q & A

What is peer grading?

Peer grading is the evaluation of students' work by their peers, which is an efficient and scalable method for assessing large classes, especially in online education and massive open online courses.

What are the potential issues with traditional peer grading tools?

Traditional peer grading tools may ignore important behavioral aspects such as the possibility of graders competing or cooperating, which could lead to inflated or reduced scores, or graders not putting in meaningful effort, resulting in unrepresentative grading.

What is the Tropican mechanism?

Tropican is a mechanism introduced to ensure that it is in the best interest of the peer grader not to engage in strategic behaviors that could compromise the integrity of the grading process.

Who developed the Tropican mechanism and the Sol Cratera tool?

The Tropican mechanism and the Sol Cratera tool were developed by a group of researchers from IIT Kanpur and UC Davis.

What does Sol Cratera mean and how does it relate to peer grading?

Sol Cratera, which means 'self grader', is a tool that uses the Tropican mechanism for peer grading, aiming to ensure fair and representative grading by students.

How does a user create a class and enroll students in Sol Cratera?

Users can create a class and enroll students by uploading a roster, which sends email confirmations to all participants.

What role does the instructor play in the peer grading process in Sol Cratera?

The instructor can choose the grading method, upload assignments and answer keys, set maximum marks for questions, upload solutions, and monitor the progress of the peer grading process.

What are 'probes' in the context of the Tropican mechanism?

Probes are a special set of papers used by the Tropican mechanism to probe the graders. They are selected from submitted assignments, graded by the instructor, and serve as the gold standard for grading.

How does the instructor monitor the progress of peer grading in Sol Cratera?

The instructor can monitor the progress of peer grading by viewing the grades awarded to papers assigned to graders and tracking the overall progress of each peer grader.

What is the purpose of the regrading request feature in Sol Cratera?

The regrading request feature allows students to request a reevaluation of their grades, which is a standard feature in many grading systems to ensure fairness and address potential grading errors.

What is the significance of the Tropican score given at the end of the peer grading exercise?

The Tropican score is significant as it represents the mechanism's evaluation of the entire peer grading exercise, ensuring the integrity and quality of the grading process.