Creating Rubrics for Assessment
Summary
TLDRThis educational video script discusses the importance and types of rubrics in assessing students' writing and speaking skills. It differentiates between holistic, primary trait, and analytic rubrics, highlighting their advantages and disadvantages. The script emphasizes the value of rubrics in providing consistent, reliable feedback to students and suggests using rubric websites for quick development, while also reminding educators to align rubrics with their teaching philosophy.
Takeaways
- 📝 Rubrics are essential for assessing students' extended responses in writing or speaking, requiring subjective evaluations.
- 🔍 Rubrics provide consistency and reliability in scoring across multiple students, helping to ensure fairness.
- 📚 Students generally appreciate rubrics as they offer clear criteria for success and areas for improvement, enhancing engagement and motivation.
- 🛠️ Teachers often have a complex relationship with rubrics, finding them useful yet sometimes challenging to perfect.
- ✍️ Primary Trait Rubrics focus on a single characteristic of student work, but may lack comprehensive feedback.
- 🎯 Holistic Rubrics offer a quick and reliable method to score student work, providing a single grade that reflects overall proficiency.
- 🔑 Holistic Rubrics can be easier to create and use, especially for high-volume assessments like placement or final exams.
- 🔍 Analytic Rubrics break down assessment into sub-skills, allowing for more detailed feedback and the option to weight different aspects of the work.
- ⏱️ Analytic Rubrics may take longer to score due to the need to evaluate each sub-skill individually.
- 💡 When creating a rubric, consider the scoring scale, the sub-skills to be evaluated, and how to write clear and distinct descriptors for each level.
- 🌐 Rubric websites can be a helpful tool for quickly generating rubrics, offering flexibility in selecting or creating sub-skills and descriptors.
Q & A
What is the primary purpose of using a rubric in teaching?
-The primary purpose of using a rubric in teaching is to assess students' performance in a consistent and reliable manner, particularly for extended responses in writing or speaking that require subjective evaluation.
Why are rubrics beneficial for students?
-Rubrics are beneficial for students because they provide clear criteria for success, allow students to understand their strengths and weaknesses, and can be motivational as they offer a sense of stability across multiple assignments.
What are the different types of rubrics mentioned in the script?
-The script mentions three types of rubrics: primary trait rubrics, holistic rubrics, and analytic rubrics, each with their own strengths and weaknesses.
What is a primary trait rubric and why is it less commonly used?
-A primary trait rubric evaluates a single trait on a piece of writing or speaking, such as persuasive ability in a persuasive essay. It is less commonly used because it does not provide a lot of feedback and does not recognize multiple aspects of an essay, like grammatical accuracy or vocabulary range.
How does a holistic rubric differ from an analytic rubric?
-A holistic rubric provides a single score based on an overall assessment of a student's work, while an analytic rubric breaks down the assessment into multiple sub-skills, each with its own descriptors and scores.
What are some advantages of using holistic rubrics?
-Advantages of holistic rubrics include speed and ease of use, reliability in scoring across multiple students, and the ability to yield a single score that is easy for administrators to interpret.
What are the potential disadvantages of holistic rubrics?
-Disadvantages of holistic rubrics include the difficulty in accurately assessing students with varying strengths and weaknesses, the lack of detailed feedback for students, and challenges in ensuring consistency across multiple markers.
Why might an analytic rubric be preferred over a holistic rubric?
-An analytic rubric might be preferred when there is a need for detailed feedback on specific sub-skills, when sub-skills can be weighted according to their importance, and when more reliable scores are needed, particularly across multiple markers.
What are some challenges in developing an analytic rubric?
-Challenges in developing an analytic rubric include the time-consuming process of scoring due to the assessment of each sub-skill separately, the difficulty in creating distinct descriptors for each sub-skill, and the potential for overlap or ambiguity in descriptors.
How can a rubric website assist in the creation of a rubric?
-A rubric website can assist by providing a platform to select from pre-determined sub-skills or create custom ones, choose from pre-written descriptors or write original ones, and generally offer a flexible and efficient way to develop a rubric quickly.
What is the importance of reflecting on the decisions made while developing a rubric?
-Reflecting on the decisions made while developing a rubric is important because these decisions, such as the grading scale and the sub-skills chosen, reflect the teacher's values and philosophy towards teaching and learning, and ultimately influence student proficiency.
Outlines
📚 Introduction to Rubrics in Education
This paragraph introduces the concept of rubrics as essential tools in assessing students' writing and speaking assignments. It discusses the importance of rubrics in providing consistent and reliable grading across multiple students. The speaker emphasizes the practicality of rubrics in scoring, their role in offering feedback to students about their strengths and weaknesses, and their utility in enhancing accountability in education. The paragraph also touches on the common love-hate relationship teachers have with rubrics and the idea that they should be continuously improved upon use.
🔍 Types of Rubrics: An Overview
The second paragraph delves into the different types of rubrics used in education, specifically focusing on primary trait, holistic, and analytic rubrics. It explains that primary trait rubrics evaluate a single characteristic of student work, which can be limiting in providing comprehensive feedback. Holistic rubrics are described as quicker to use and more reliable, offering a single score that can be beneficial for administrative purposes. However, they may lack in providing detailed feedback to students. The paragraph also mentions the ease of creating holistic rubrics compared to analytic ones, which are not discussed in detail here.
📉 Challenges and Uses of Holistic Rubrics
This paragraph explores the advantages and disadvantages of holistic rubrics in more depth. While they allow for quick scoring and consistency, they can be less reflective of individual student abilities due to their collapsing of multiple criteria into a single score. The challenge of ensuring consistency across different markers is also highlighted. The paragraph suggests that holistic rubrics are particularly useful for high-volume assessments like placement tests or final exams, where speed is crucial.
📝 Analytic Rubrics: Detailed Feedback and Scoring
The fourth paragraph introduces analytic rubrics, which assess multiple sub-skills individually, providing specific feedback to students. It discusses the advantages of analytic rubrics, such as the ability to weight sub-skills according to their importance in a particular context, and the potential for more reliable scoring. The paragraph also addresses the challenges of developing analytic rubrics, including the time-consuming process and the difficulty of creating clear, non-overlapping descriptors.
🎯 Creating Effective Rubrics
The final paragraph provides guidance on creating effective rubrics, emphasizing the importance of decisions made during their development, which reflect the educator's values and approach to teaching. It discusses the selection of a scoring scale, the identification of sub-skills to evaluate, and the crafting of distinctive descriptors for each sub-skill at each proficiency level. The paragraph also suggests using rubric websites as a starting point for developing rubrics and the importance of pilot testing them before high-stakes use.
Mindmap
Keywords
💡Rubric
💡Assessment
💡Consistency
💡Feedback
💡Accountability
💡Primary Trait Rubric
💡Holistic Rubric
💡Analytic Rubric
💡Descriptors
💡Proficiency Levels
💡Reliability
💡Scoring Scale
Highlights
Importance of rubrics in assessing student writing and speaking for consistent and reliable grading.
Different types of rubrics: holistic, primary trait, and analytic, each with unique strengths and weaknesses.
Holistic rubrics provide a quick and consistent scoring method but may lack detailed feedback.
Primary trait rubrics focus on a single characteristic but may not provide comprehensive feedback.
Analytic rubrics offer specific feedback on sub-skills but can be time-consuming to score.
Students generally appreciate rubrics as they understand criteria for success and receive clear feedback on strengths and weaknesses.
Rubrics can be motivational for students, especially when they are familiar with the criteria used for evaluation.
The development of a rubric should be an ongoing process, with adjustments made after each use to improve its effectiveness.
The choice between holistic and analytic rubrics depends on the teacher's assessment needs and practicality.
Creating a rubric involves decisions that reflect the teacher's values and philosophy on language teaching.
Decisions on scoring scale and sub-skills to be evaluated are crucial in rubric development.
Descriptors in a rubric should be clear and distinctive for each proficiency level and sub-skill.
Pilot testing a rubric can provide valuable feedback for refinement before high-stakes use.
Rubric websites can be a helpful starting point for quickly developing a rubric, especially when time is limited.
The use of rubrics enhances accountability in education by providing students with detailed feedback on their performance.
Challenges of holistic rubrics include difficulty in assessing individual student abilities when criteria are collapsed into a single score.
Analytic rubrics allow for weighting of sub-skills according to their importance in the context of the course.
The development of a rubric is not value-neutral and should reflect the teacher's educational values.
Rubrics should be chosen based on the type of assessment and the desired level of feedback for students.
Transcripts
[Music]
[Applause]
[Music]
as soon as you begin to teach very
shortly after that you begin to think
about what you need to do in order to
assess or to test your students and as
soon as you have them doing any kind of
writing or any kind of speaking you need
to think about how you're going to grade
that in a consistent way across multiple
students and at that point you probably
need to start thinking about how what
kind of rubric to use and how to create
a rubric and that is what I want to
speak to you about now
[Music]
you may have a fairly good idea about
what a rubric is generally we're exposed
to them in our teacher training and you
can see an example of a holistic kind of
rubric it looks like a table with a
score down the first column and then
descriptors of criteria in the main body
of the table and this is an example then
of a holistic rubric but there's other
types of rubrics and you may want to use
a different type of rubric for a
different type of test task so we want
to get to the point where you can match
the test tasks with the type of rubric
that you need to use when do we use
rubrics you'll want to use a rubric
anytime you have an extended response
from a student in speaking or in writing
that requires a subjective evaluation as
teachers we often really like to use
rubrics for a variety of reasons one is
when we're looking at extended responses
made by our students it's actually very
practical to have a rubric it makes your
scoring a little bit easier it ensures
that your scoring is consistent or we
could say reliable across multiple
students it does provide some type of
feedback to students usually because
students are able to look
the descriptors of the proficiency level
and get an idea of their strengths and
weaknesses and so it enhances it's it's
a way to provide accountability to our
students because we're really providing
them with as much feedback as we can so
teachers generally like to use rubrics
often though teachers have a love-hate
relationship with their rubrics they
like to use rubrics but perhaps not this
one or they can find problems with a
particular rubric and so when you start
to develop or thinking about using your
own rubric don't think ever that your
rubric is absolutely completely finished
you may want to tweak it as you go along
after each use to make it and improve it
make it a little bit better and better
but generally teachers really like
rubrics what about students do you think
students like rubrics the answer to that
question is usually yes students do like
rubrics again because they can see the
criteria for their success they can see
their strengths and their weaknesses
they usually provide quite a bit of
feedback to students often students are
more engaged when they know the criteria
upon which you're basing your
evaluations and it can be motivational
for students and it can if they have
worked with a rubric that you've been
using the rubric before and they're
familiar with it it can provide them
with a real sense of stability across
multiple assignments so yeah usually
students like rubrics true
[Music]
there are different types of rubrics
that teachers frequently use when they
evaluate student speaking or writing and
I'm going to mention three of them now
one is a primary treat rubric the other
is a holistic rubric and the last one is
an analytic rubric and each of these
types of rubrics have their own
strengths and weaknesses that you should
be aware of a primary trait rubric is a
rubric that just evaluates a single
trait on a piece of writing or speaking
so it is the rubric that's probably the
least commonly used and you'll be able
to see why as I get into my explanation
so let's say you have asked students to
write a persuasive essay and you decide
that the main characteristic the prime
characteristic that you want to evaluate
in that assignment is the persuasive
ability of that particular student and
so you would provide a single score that
is dependent upon it whether that
student managed to persuade you of their
point of view or not so you can see a
primary trait rubric then refers to
evaluation of a single trait that you
have determined to be the most important
for that particular thing for that
particular assignment and usually it's a
fairly simple scale of 0 to 3 either you
were not persuaded or you were sort of
persuaded or you were definitely
persuaded I guess that would be a scale
of 0 to 2 so it's quite a simple rubric
and you can see that that type of rubric
would have some challenges with it in
the sense that it doesn't provide
students with a lot of feedback and it
doesn't recognize that well an essay
might be quite persuasive it may
actually be weak in terms of its
grammatical accuracy or its vocabulary
range or even its organization and so
although we mentioned primary trait
rubrics because people do use the
usually they're not used very frequently
so when we want to talking about the
other two types of rubrics now you can
see on the screen an example of a
holistic rubric that would be used to
score student writing you can see in the
first column there is a series of grades
that range from A to E and in the main
table are the descriptors that describe
proficiency levels at each of those
grade levels if you take a look at the
top row the next to the score of a or
the grade of a you can see that the
descriptors include excellence in
content as well as organization
vocabulary range as well as spelling and
punctuation and there's even a comment
about coherence in there so that's a lot
of information to contain in a single
description of proficiency and you can
see that this perhaps leads to the
weakness of this particular rubric there
are advantages to holistic rubrics and
you may be able to guess that one of the
advantages is that they're quite fast to
use in terms of the scoring that you can
accomplish with them is fairly speedy so
and as you get more practice with using
these types of holistic rubrics you get
to have a very good feel about what a
piece of student writing looks like at
each grade and you can play students
quite quickly in a grade level ABCD or e
and you can move relatively quickly
through your student writing so they are
fast they are fairly reliable in that
they allow you consistency in marking
across multiple students and these are
really good reasons to use holistic
rubrics the other benefit of holistic
rubrics is that they yield a single
score this is very useful actually if
you are going to give that score to an
administrator really an administrator
really wants a single score that's all a
one just tell me how good that student
is so that I can place them or that I
can tell either admit them or not admit
them and so there's always a push from
the administrator to have a single score
that is easy to interpret and holistic
rubrics generate that you can see that
based on their descriptors you will have
a student will have a single score and
so you might use a holistic rubric for
that purpose the other thing about
holistic rubrics is that although most
rubrics are challenging to create a
holistic rubric is perhaps a little bit
easier to create than an analytic rubric
so if you're thinking of developing your
own rubric then a holistic rubric might
be a good way to start it wherever
possible it's much easier to work with a
rubric that perhaps some of your
colleagues have generated or that has
been generated through research so use
those if you can and if not feel free to
generate your own but just accept the
fact that it may take a little while and
again the holistic rubric may be a
little bit easier to start with than an
analytic rubric holistic rubrics are
also very useful for situations where
you need to score a lot of tests or
exams fast so often those involve
placement tests or final exams where you
may have large numbers of students
completing an assessment or a test
altogether and you need to process their
work quite quickly in those situations
holistic rubrics are good choices just
as there are advantages with holistic
rubrics there are also disadvantages and
you may have guessed by looking at that
example of a holistic rubric that one of
the challenges with them is that they
collapse a lot of criteria into a single
grade so what happens if you have a
student who's got really strong content
but quite poor organization it's hard
then to assess them accurately with
simply a holistic rubric because the
holistic descriptors group all of those
things into a single unified score so
sometimes you might find with your
holistic rubric that you have two
students who are really quite
friend inabilities one has strong
content and weak organization and the
other has really good grammatical
accuracy but poor content and they may
end up with a similar grade and so the
holistic rubric is less able to reflect
the differences between those students
because of the way it's constructed
holistic rubrics also don't provide a
lot of feedback to your students in
terms of their strengths and their
weaknesses again if you think of that
student who's got really strong
grammatical accuracy and really weak
content they're not going to be able to
see that from the holistic rubric so
that's a really important weakness if
you're scoring a test that has that
requires multiple scores or multiple
markers you may also find that it's
quite difficult to get everybody marking
in the same way using a holistic rubric
it's a bit of a challenge to ensure
consistency across markers and so that's
a real shortcoming for holistic rubrics
as well let's have a look at analytic
rubrics then on the screen you can see
an example of an analytic rubric you can
see the example there it's a table
that's actually empty analytic rubrics
once the content is filled in in the
table the descriptors are added in
analytic rubrics can often be quite long
and so it's challenging to get
everything on one screen but you can see
the outline for it here like a holistic
rubric analytic rubrics have the grade
score the grade scale in the first
column and so you can see that here
across the top though they have multiple
columns and each column has a sub skill
that broken out and then descriptors for
each sub skill so you can see for
writing you might have a column for
content one for organization one for
grammar or for more accuracy another for
vocabulary or any other sub skill that
you decide is important to assess
in your context and although we were
able to add descriptors in this
particular example because it would have
made it too long the middle part of the
table usually contains descriptors of
each of the sub skills at each of the
proficiency levels just like holistic
rubrics analytic rubrics have advantages
and disadvantages so let's talk about
the advantages first one of the most
important advantages of an analytic
rubric is that it provides very specific
feedback to students so they can clearly
see whether they have strengths and
content or weaknesses and organization
strengths and vocabulary range or
weaknesses and spelling or punctuation
and this is really important feedback
for students to get so in that sense
it's very helpful to the student the
other thing a teacher can do with an
analytic rubric is because the sub
skills are broken out the teacher can
actually weight the sub skills according
to how important they think those are
and this is an option that's not
available to you when you use a holistic
rubric so for example let's say you're
teaching a grammar course where you ask
students to write quite a bit in their
assessment of their writing you may use
a rubric but it would be completely
understandable if you weighted the
grammar component of that particular
task more heavily than content and
organization because that would match
very closely with what you had been
teaching students that would make sense
conversely if you had an essay writing
kind of course where you were focusing
mostly on different rhetorical patterns
you might choose to emphasize content
and organization and de-emphasize the
grammar or accuracy part you may you may
not but you have that option when you're
a teacher using an analytic rubric
another advantage of an analytic rubric
is that it produces more reliable scores
than a holistic one particularly across
multiple scores but also possibly just
for your own scoring purposes because
you're assessing each sub skill skill
individually your
to make a judgment on one particular sub
skill before you move on to the next
particular sub skill so you can evaluate
content separately from organization
separately from accuracy another
advantage of analytic rubrics is that if
for example you're using them for
research purposes the analytic rubric
then gives a really good picture to the
researcher of the strengths and
weaknesses of your student that's sort
of an unusual circumstance but if you
are involved in a research project it's
a good idea probably to use an analytic
rubric of course there are disadvantages
to analytic rubrics as well and one of
the bigger disadvantages is that it
takes a longer time to score because
you're weighing each or you're judging
each sub skill separately it may take
you a little bit more time than it does
if you were using a holistic rubric you
may also find that analytic rubrics are
harder to develop if you have to build
an analytic rubric yourself is probably
a little bit more challenging because
you have to write more descriptors for
analytic rubrics than for a holistic
rubric and when you're writing those
descriptors you may have a challenging
time avoiding overlap between one
descriptor and another and ambiguity so
that's always a challenge when you're
creating an analytic rubric another
challenge with an analytic rubric is
that it may not allow you to assess the
overall effect of a text and by this I
mean that you may actually have a
student who produces a writing text that
is fairly weak in terms of grammar the
accuracy is quite poor than making lots
of mistakes and yet overall boy you
really appreciate what they've done
they've created a text that's creative
that's really interesting it may not
have superior content but perhaps
somehow it's very appealing due to its
rhythmic quality or almost poetic
license perhaps that the student has
used and so if you are very analytic
with the rubric if you have an analytic
rubric and you
very analytic when you look at pieces of
writing like that you may not actually
be able to capture the overall effect
with an analytic rubric so when it comes
time for you to create a rubric you
probably want to choose between a
holistic rubric and the analytic rubric
perhaps your ultimate choice will be
determined by reasons of practicality
and the reason for your assessment so if
you've got a lot of tests to score
you're probably going to choose a
holistic type of rubric if you have
fewer scores to do fewer students to
score and you're really interested in
getting good feedback to those students
you may choose an analytic rubric all
right so let's look at the framework
that we looked at before when we were
talking about analytic rubrics and for
each area of the rubric you need to make
decisions and your decisions really
reflect your philosophy or your approach
to teaching and learning a second
language
so keep that in mind whenever you're
developing a rubric that your decisions
then reflect your values so they're not
value neutral and that's important to
remember all right so your first
decision then when you create a rubric
is to decide on your scoring scale do
you want a lot of divisions or just
smaller number of divisions do you want
to run from 0 to 4 or 1 to 6 or 0 to 9
or 10 research shows that if you have
quite a few divisions it gets quite
difficult to write distinctive
descriptors for each sub skill so
oftentimes it's better to keep the
scoring range a little bit lower so
let's say you may want to go 0 to 6
where you start the scoring scale is
important
perhaps you started at 0 and that would
be a grade that you would assign to
students who have 0 production but
perhaps you want to recognize that
students who even put a few words down
even if they don't get to write an essay
or a structured essay even students who
put a few words down are deserving of at
least 1 mark in that case if that's your
philosophy then you would start your
scale at 1 and the number of divisions
that you have then really reflect how
clearly you can write distinctive
descriptors for each level to create an
analytic rubric your next decision
relates to the sub skills that you're
going to evaluate and they're going to
write those across the top of the
columns so you've got to decide what
you're going to evaluate generally if
you're evaluating writing you're going
to have a column for content 1 for
organization and 1 for grammar accuracy
or form but there are lots of other
different sub skills that people may
find pertinent to their teaching
contexts so for example have you been
really encouraging your students to
watch their spelling and check their
spelling then you may want to add a
separate column for spelling that might
be quite important to you and to what
you're trying
teachers students or you may really want
to emphasize vocabulary range you've
really been building your students
vocabulary and you want to see them
applying that in their written work so
it may be really useful for you to have
another column there where you attempt
to assess their vocabulary range so
whatever you pick do across the headings
of those columns that reflect the sub
skills that's your choice
and it's your it reflects your values
what you value in writing proficiency or
in speaking proficiency in your students
when developing a rubric you'll also
need to spend quite a bit of time
developing the descriptors that go in
the middle of the table so you want to
write distinctive descriptors at each
grade level for each sub skill and that
can be really challenging but it's
really really important one good idea
when you're developing an analytic
rubric is to draft one and if you can
pilot test it either with some old
assignments that you might still have
around that you've kept or you may ask
some of your colleagues to have a look
at it and make suggestions or you may
even run it as a test with some of your
students and ask them if they could work
with this rubric or make suggestions or
have them try and do a peer assessment
with it it's good to try and get
feedback on an analytic rubric before
you actually use it for a higher stakes
situation it's difficult to develop an
analytic rubric and you may find that as
you go along and you use it assignment
after assignment you may want to change
it and update it so I always think for a
mom of my own rubrics that they're never
really in a finished State they're
always trying to push them to be a
little bit more accurate in their
descriptors a little bit to reflect a
little bit better the skills that I want
my students to learn one thing that you
may find really useful when you're
developing rubrics either holistic or
analytic rubrics is a rubric website
there are some really good websites on
the internet that help create rubrics
and you can go to one of these most of
them are free generally they require
just sign on with your email and to
create an account but then you don't
need to pay for them and you can if you
want select from a group of
predetermined sub skills or you can
create your own sub skills you can then
select from pre-written descriptors or
you can write your own descriptors
there's a fair amount of flexibility and
how these rubric websites work so you
may find that they're really useful
particularly if you're in that situation
very common to many teachers it's the
night before and you wanted to hand back
students assignments the next day and
you really don't have a rubric that
you're comfortable with in terms of
assessing the writing rubric websites
are not a bad place to start again you
may not want to finish there or just
leave it at that but they're really a
pretty good place to start to develop a
rubric quickly to conclude then when you
are testing anytime you have an extended
student response in either speaking or
in writing you're going to want to use a
rubric and you probably want to choose
between holistic or analytic rubrics
although you might consider primary
treat for those situations where you
want a really fast smart really really
speedy but you're probably going to pick
between a holistic and analytic rubric
and just remember as you're developing
your rubrics that all of your decisions
the grading scale and the sub skills
that you choose to evaluate and how you
write at your descriptors they all
reflect what you value in student
proficiency
[Music]
[Applause]
[Music]
Weitere ähnliche Videos ansehen
5.0 / 5 (0 votes)