A common challenge for instructors teaching large classes is the management of graduate teaching assistants. These teaching assistants are often tasked with grading in large classes, but they come to that activity with vastly different conceptions of what effective grading looks like and how one can grade effectively in a reasonable amount of time. Likewise, teaching assistants come to our classes with different teaching skill sets and life experiences.  Some of them are mature, effective teachers, while others are preparing to teach their first class.  In our large classes, these issues are often amplified by the large number of TAs that we may require.  How do we ensure that our TAs are all on the same page and doing their jobs well? 

CCBY Wilsman, A. (2013). Teaching Large Classes. Vanderbilt University Center for Teaching.

CCBY Tips to Address Assessment Consistency with Many Assessors. Centre for Teaching Excellence, University of Waterloo. 

Grading with TAs 

Grading 

A common undergraduate complaint in large classes is with regard to inconsistency in grading.  Most instructors will recognize the refrain, “My TA is an unfair grader! Can I change sections?” Indeed, it can be frustrating for undergraduates who believe that they are the victim of the “tough grader,” and are receiving worse grades than their friends despite handing in comparable work. So how do we ensure consistency and mitigate undergraduate charges of unfairness? 

Have regular grading meetings! The best way to promote grading consistency among our TAs is to meet as a group soon after collecting an exam or paper. If one is grading essays, identify and photocopy an exemplary essay, a few mediocre essays, and a poor essay and distribute these essays to each member of the group. Prior to the meeting, have each TA grade and comment upon these essays. At the meeting, go through each essay one-by-one. Ask each person what grade they gave to each essay and why.  Ask them about the best and worst aspects of each piece of writing. Such a meeting provides a wonderful opportunity for our graders, especially our inexperienced graders, to think about how they’re approaching their grading. It can serve to calibrate expectations for the exam or paper. The meeting can also serve as a forum for us to explain our expectations for the exams or papers. It is unfair to assume that our TAs will simply know what we’re looking for on any given exam question or paper topic. 

Use grading rubrics. A carefully designed grading rubric can both minimize the amount of time spent grading, an important consideration in large classes, and serve as a common standard for our TAs. We can even enlist TA support in constructing a grading rubric. Such an exercise can be valuable to TAs because it facilitates the grading process, but it also gives them an opportunity to play a major role in student assessment, a valuable experience for those TAs who hope to teach courses of their own at some future time. It also gives us a new and unique perspective on class exams, papers, and assignments that may ultimately enrich the course. 

Divide up grading sections. We can better ensure consistency by assigning different grading sections to different TAs. This is more challenging with essays, but is a common approach for exam-grading.  What this technique entails specifically depends on the makeup of our exam, but for example, perhaps one TA grades the short-answer section, a second TA grades the first essay, and a third TA grades the second essay.  While there still may be some inconsistency in the “harshness” of grading between sections, with this method, students can hardly argue that their particular grader is tougher: everyone’s exam is graded by the same graders! 

Managing TAs Who Lead Discussions, Lab Sessions, and Review Sessions 

Before we get to know our undergraduates, we ought to get to know our TAs

Know Your TAs.  Just as with grading, TAs come to discussion-leading with different levels of expertise.  Some will be at home in the classroom. Others will be terrified to speak in front of their students.  It’s a good idea to get a gauge on this in the weeks preceding the semester so that we can give our TAs the appropriate level of support. Some may be independent-minded and will desire considerable control over what happens in their classrooms, and others may require strong guidance. Thus, before we get to know our undergraduates, we ought to get to know our TAs. 

Hold regular meetings.  Should we have TAs that require a strong support system or even if we don’t and we want to maintain some control over discussions, lab sessions, and review sessions, regular weekly or bi-weekly meetings can be valuable. These meetings can serve many purposes. We can use this time to go over important concepts and course content with our TAs who likely don’t have our expertise.  We can also use this time as a “check-in” period to get a sense of how the course is going for our TAs and undergraduates alike. Professor/TA meetings can also be a forum in which we provide TAs with handouts or discussion guides to help facilitate their class time. Ultimately, how much control we want to exert over our discussion sections, lab sessions, or review sessions is up to us, but setting aside a time to meet with our TAs is valuable because it provides professor and TA alike with a support structure in which everyone can talk through issues relating to the class. 

In most classes, large or small, grade complaints are inevitable. However, the issue can become more pronounced when a couple of upset students becomes a dozen or more. How can we best deal with grading complaints? 

Have a formalized system in place.  Instructors of large classes approach grade complaints in a variety of ways. Some insist that undergraduates come directly to them with their concerns. Others suggest that undergraduates speak to their TAs first before consulting the professor. Still others give full authority to their TAs to handle all grade complaints. The important thing is that we have a formalized system, preferably outlined in our syllabus. Students should know what is expected of them, and what their options are if they feel that they have been graded unfairly. Tell students upfront what the protocol will be. 

Require complaints to be written out and submitted.  One common technique to avoid frivolous grade complaints is requiring a written explanation of the complaint at an early stage in our protocol. Oftentimes, upon starting this piece of writing, undergraduates with a visceral reaction to a bad grade will see that the grade was deserved. By requiring this piece of writing, we’re also forcing students to confront the written comments on their exam or paper. Sometimes, students simply see the bad mark and seek out the instructor, rather than reading and mulling over grader comments. 

Institute a 24-hour rule.  Another way to ensure that students are carefully considering the grade and comments and aren’t simply going with a visceral reaction is to have a 24-hour rule. What that means is that students are required to take 24-hours before contacting the TA or professor with a grade complaint.  This 24-hour period often serves as a “cooling off” period in which students can read and think about grader comments. 

Tips to Address Assessment Consistency with Many Assessors 

All instructors want to assess fairly, but doing so can sometimes require extra effort, especially when a large number of students are being graded by multiple assessors. Consider following these guidelines the next time you find yourself working with a team of assessors. 

Prior to the assessment 

1. Communicate expectations, requirements, and guidance to students: 
2. Read the cover page of the assessment; 
3. Read the whole question before starting on an answer; 
4. Complete all requirements; 
5. Where appropriate, follow approaches modeled earlier in the course for similar questions; 
6. Aim for clear, complete, and well-organized responses. 

Before the assessment session 

1. Construct a rubric or grading key as soon as the assessment is designed. 
2. Determine and arrange logistics for assessment day, including the location; expected duration of the session; snacks and coffee, tea, or juice for especially long sessions. 
3. Allocate assessment components or questions to assessors such that they will complete their allotted assessments roughly at the same time. 
4. After the test has been completed by students, but before the assessment session, perform a sample assessment of up to 10 tests (either randomly or choosing particular students’ assessments) and adjust the rubric or assessment key (to possibly include alternative solutions). 
5. After testing is completed by students, but before the assessment session, provide the test to the markers for them to review prior to the marking time.  

At the assessment session 

1. Use control and tracking sheets to track which assessor marked which test (using a rubric in LEARN will track assessors automatically) and for statistics on mark breakdown within an assessment. 
2. If possible have only one assessor for each part of every assessment. 
3. Seat assessors together in one location to facilitate communication between groups of assessors. 
4. Have each group of assessors mark a sample of up to 5 exams for their assigned part, then discuss results of their sample, and potentially adjust the assessment key (the lead assessor or instructor should be part of this conversation among the groups or assessors). 
5. Have assessors forward problematic responses (such as illegible handwriting or ambiguity due to English as a second language issues) to the instructor to assess. 
6. During the assessment session, have the instructor review samples of marked exams for consistency/issues between assessors. 
7. After all of the assessments have been completed, have each assessor or group of assessors identify and record common gaps. (This information can help the instructor refine the learning activities the next time the course is offered.)  

After the marking session 

1. Use three recorders to enter the grades: one to read, one to enter the grade, and one to watch. 
2. Review assessments with low marks, especially for final exams where the overall course mark is near pass/fail or minimum program requirements. 
3. Look for and investigate surprises such as a wide range of assessment marks, bearing in mind the overall academic performance of the cohort of students taking your course. 
4. Confirm and communicate to students the basis for an appeal, gather assessments deemed to qualify for review, and review them all at once, if possible. 
5. Determine what will be communicated to various groups of students based on their performance on the assessment.

Resources

Wilsman, A. (2013). Teaching Large Classes. Vanderbilt University Center for Teaching.  

Tips to Address Assessment Consistency with Many Assessors. Centre for Teaching Excellence, University of Waterloo.