The Holy Grail of Teaching: Better Learning Outcomes with Less Prep-Work and Grading

Introduction
This post is directed primarily at teachers but I think students and parents alike can benefit from reading it in order to understand what goes on behind the curtain. I'm going to focus on course design rather than teaching technique, although the two overlap. Designing a course is perhaps one of the most complex and challenging tasks under the sun because it requires balancing a large number of competing objectives. Let me explain.

1. Depth vs Breadth: Every class period you spend going deeper into a sub-topic is one class you take away from the breadth of the course content. And vice versa.

2. Reinforcement vs New Content: Every class you spend reinforcing previous content is a class you don't spend on new content. And vice versa.

3. Predictability vs Flexibility: For some reason, students panic or perceive instructors as disorganized whenever the original syllabus changes. However, different cohorts will find different units interesting (and uninteresting). Ideally, we want to tailor a syllabus to a cohort; that is, if a cohort finds a unit particularly interesting there are good pedagogical reasons to extend the unit. Similarly, if a cohort finds a unit boring, you want to be able to cut it short sometimes. The trade off, however, is that every time you change the syllabus, students panic or perceive you as disorganized . On the flip side, you maintain rigidity at the cost of students not getting additional time on what they enjoy or shortening units they don't enjoy.

4. Assessment and Incentive vs Time Grading: This is a big one. Here's the reality: For a variety of reasons I won't get into--because I'll start ranting like a lunatic--most students have been conditioned to see little if any intrinsic value in doing any school work. This includes doing the readings before class, any kind of written activity, or self-assessment activity. In other words, if it doesn't count towards their grade, most students either won't do it or will do a crappy job of it. However, here's the thing about learning a new skill or new content: It ain't happening without practice and repetition. To quote Aristotle, "We are what we do repeatedly. Excellence then is not an act but a habit." So, how do we get students to get the practice and repetition they need for learin' to happen?

Easy fix, you say. Just make them do lots of assignments. Problem: students hate what they call "busy work." Students must perceive assignments to be both relevant and worthwhile for them. Instructors must give careful forethought to the content of assignments, how they fit in to course objectives, how they relate to other course material, and how much weight they should be accorded as a percentage of the course grade. When weighting an assignment or task you must consider the direct relationship between weight and student motivation. You must also constantly refer back to your course objectives: What do you want your students to be able to know and do by the end of the semester? This should inform your weighting.

The major trade off with assessments involves grading time. It's all fine and dandy to assign regular homework or assignments but someone has to grade those. And that someone also has to prep classes and probably wants a faction of a life outside of their job. And if that someone is a grad student, they also need to do coursework, work on their dissertation, submit to journals, submit to conferences, attend colloquium talks, attend various committee meetings, etc... You get the point.

Anyhow, in my method described below, I'll explain how to juggle these competing ends in a way that delivers better learning outcomes but with less prep-work and grading than you're probably doing.

Teaching College Kids in the Twenty First Century
Check it. Yo, yo. What's up? Immabout to drop some knowledge on y'all. <----Talk like that a lot.

Outside of the classroom more students than ever are working part-time or even full time. Depending on the source, around 5/6 of students work at least 19 hours/week. When they aren't working, many of them aren't doing school work. And if they are doing school work, many of them are simultaneously texting and watching cat videos.

Whatchugonnadoabouddit?

• Choice 1: Whine and complain about students "these days." 
  
• Choice 2: Only teach to the minority of students who are highly motivated and/or don't have jobs. This may be combined with Choice 1.
• Choice 3: Accept that you are powerless in the face of broad sociological trends and adapt your teaching/syllabus accordingly such that you are able to reach the average student.

I'm not going to convince you which you ought to choose. Allz imma say is that I chose 3. And immabout to explain what that choice means in practical terms.

Most importantly, it means that with a few exceptions you must make class time for whatever skills or knowledge you want your students to acquire. Lemmi add a few details to that.

There are different levels of knowing which can be sliced and diced various ways. The three broad categories I have in mind when designing my courses and lessons are: (a) Basic comprehension, (b) application, (c) theory-level. There are other ways to conceive of levels and kinds of knowing--this is simply how I do it. Let's look briefly at what each means.

Basic Comprehension: A student has basic comprehension of a concept or argument if they can reproduce it. For example, they demonstrate basic comprehension when they can answer questions like: What does happiness mean for Aristotle? What does Locke think the purpose of government is? How does MLK distinguish between laws we should follow and those which we may permissibly break? Answering these questions doesn't require deep understanding but it demonstrates basic (superficial) knowledge of course content and themes. There is an even lower level of understanding which is recognition. This kind of "knowledge" is tested on multiple choice tests. The student needn't be able to recall or express the information on their own--only recognize it when presented to them. Pick the level you want your students to achieve and build in-class activities that bring them to this level. Basic comprehension is rarely the final goal; however, it's a necessary rung on the ladder to the other levels.

Application: The next level of understanding requires that students be able to apply new concepts, arguments, or skills to novel cases. For example, I might present students with a famous literary or movie character and ask them to evaluate whether Aristotle would call this person happy. Or I might ask them to evaluate whether Locke would consider a certain political revolution or law to be justified. Or I could present them with a particular law and ask them whether MLK would recommend we follow or ignore the law. In all cases I'd require them to justify their answers by appealing to the original author, otherwise there's no way to distinguish between lucky guesses and understanding.

Theory-Level: At the level of theory students begin to understand the various theoretical trade-offs and implications of different views. They compare theory to theory and draw logical implications of theories. This is requires a very high level of understanding and can only be reached after the first two have been firmly established. I find the best way to develop this level of knowledge is to get students to 'toggle' back and forth between theoretical frameworks. For example, I might present a case and ask them how different theories would appraise it. Then I'd ask them to evaluate the relative costs and benefits of the differing appraisals. (More on this below)

Whatever level(s) I want my students to attain, I must create in-class activities that foster those levels of development. Why? Because, if you accepted Choice 3 above, students are not going to do it (well) outside of the classroom. Now, you can go back to Choice 1 and whine and complain that they should. But guess what?

They. Aren't.

So you decide. Given that most students--culpably or not--are not going to engage in deep learning outside of the classroom, what level of learning do you want them to possess by the end of the semester? Pick it and stick it into your syllabus; that is, set aside class time for developing that level of knowledge.

The Bottom Line: The days of lecturing then sending students home with readings, exercises, and assignments is over. A student taking a full-load and working full time doesn't have time or make time to put in the deep concentration required for deep knowledge acquisition. Knowledge acquisition (all levels) must now be deliberately built into classroom activities otherwise it won't happen for most students. Stop whining and accept the new world order. Thanks, Obama.

The Basics
To repeat so far: All good courses begin with a firm understanding of the course objectives. You need to decide BEFORE designing the rest of your course what the students ought to be able to do and know, and the level at which they know it. This will help mitigate (but not entirely eliminate) some of the above competing trade offs you have to make. For example, if I want them to acquire a particular skill, then I have to build doing that into the syllabus. This means I'm going to reduce some of the content in order to make time for skill acquisition. To explain a bit more about how course objectives help 'set' your syllabus, I'll begin by describing common pitfalls.

On the first day of class, you read through the course syllabus, skimming over the course objectives/learning outcomes section. Or maybe you even spend a little time explaining each. What happens next? With the exception of the first week, for the rest of the semester you never mention them. Then, you are shocked! shocked! I tell you! when at the end of the semester you students fail to meet these outcomes.

Rule #1: Build in and reinforce your chosen learning outcomes. As I've said, one of my course objectives is for students to be able to interpret and argue from various competing positions. That is, I want them to learn how to see an issue through eyes that are not their own and to formulate arguments from that perspective. How to do this?

First, in each lecture, anytime an issue or case is presented, I ask the class to tell me what previous authors would have said. Then I ask them how another author would respond. This can be done through group work, take-home assignments, or soliciting volunteers. It's also important to model the skill yourself so students have template. The point is, anytime a situation arises where a course objective can be realized/practiced, we do it.

But here's what usually happens. Teachers fixate on getting through the material. "I can't stop anytime students have a chance to occupy different points of view, I'll never get through the material." This is what I mean by designing your course from the objectives.  If a core objective is for students to be able to argue from competing perspectives then opportunities to do this shouldn't be interfering with a well-designed syllabus. The syllabus should be built to allow students to practice exactly this thing!!!111!!!--not just to bulldoze through a set of readings. The fact that the pace of readings "interferes" with your course objectives should tell you to go back a revise the syllabus. Begin with objectives then decide on number of readings.

Build your objectives into the assignments and classwork. You can't just tell students "here are your objectives for the course" then magically expect them to achieve them. Where on God's green earth did someone ever acquire a new worthwhile skill without close supervision, repetition, guidance, practice, and critical feedback? It takes a lot to acquire a new cognitive skill. You are teaching someone to think differently. That means you're fundamentally changing the way their brain operates. This does not happen overnight and it certainly doesn't happen by accident.

All ranting aside: You must build time into your syllabus for your students to practice and develop the course objectives. Like I said, it does not magically happen. So, if you want students to be able to reconstruct arguments, you must build class time into your syllabus to do this. Also, the fact that you build it into class time in sends a message to students that it's important--it's not just an afterthought. In the next section, I'll give more concrete suggestions on how to do this and explain why it needs to be part of class time.

Rule #2: One of the wisest things I've ever heard comes from Paul Woodruff. In his excellent book, The Ajax Dilemma, he says "If you want to know what an organization values, look at what it rewards." Most students will only do what they are rewarded for and they will do it in proportion to the size of the reward. So, reward them (i.e., give them points) for the things you want them to do and how much you want them to do it. Want them to do the readings before class? Find a way to reward that. Want them to show up to class? Find a way to reward that. Want them to improve their writing? Reward the improvement not just the writing (more on this below). If you don't reward something, from the point of view of students, it's just "busy work." Reward the things you value for your course (which should be the course objectives).

Yeah, I know students are supposed to be intrinsically motivated by the beauty of knowledge and all that nice stuff. But if we want to reach as many students as possible we need to be a bit Machiavellian or Pavlovian--pick your metaphor--in our approach. Below I will give you specific ways that I have found to be successful in rewarding the various outcomes I seek.

The Holy Grail Method: How to Get Better Learning Outcomes with Less Prep-Work
Where I teach, most classes meet 3 times a week for 50 minutes. If you teach a course that meets twice a week, you can modify my method by making every 3rd or 4th meeting the activity day.

Overview:

1. Mondays and Wednesdays are new content (i.e., lecture).
2. Fridays are reinforcement, application, and critical appraisal.
3.  5 minute, 5 question multiple choice auto-graded online quizzes at the beginning of each lecture class.
4. Quizzes should involve the core elements of the reading. Lectures should answer the questions on the quiz.
5. Fridays are in-class group work.

• Each activity sheet contains three main sections: Basic comprehension, application to novel cases, theory-level questions. Ideally, the sections are related.
• For all exams, questions are selected exclusively from the Friday work-sheets. This incentivizes effort and care in doing them.
• Only grade 3 or 4 questions selected at random (same for all groups) for the assignment's grade. This reduces grading time but ensures members work collectively and check each other's work.

Grading:

• Reading Quizzes: 25% of final grade. (lowest 3 scores are dropped)
• In-class group assignments: 25% of final grade.
• Midterm: 15% of final grade
• Short paper: 15% of final grade
• Final paper: 20% of final grade (2x10% each peer editing sheet; 20% responsiveness to peer reviewers; 60% the final version of the paper).

Getting your Students to Read: Any way you slice it, part of a good education at any level should improve reading skills. And how do we get better at anything? We do that thing, and we do it at a level slightly beyond our existing level. The? A? problem is many students nowadays don't read. [Shakes fist in the air]. How do we get them to read? Well, what gets rewarded gets done.

At the beginning of each class for which there is an assigned reading, give an online (i.e., auto-grading) 5-question multiple choice quiz. Make the quiz password protected (Canvas and Blackboard have this feature). Put the password up on the board when you enter the classroom. Doing this also solves attendance problems since you can only do it in class. You don't need to take attendance because students will show up if there are points at stake. Boom goes the dynamite.

Aside: I drop the 3 lowest scores. This allows me to avoid dealing with determining the legitimacy of absences. You get 3 free low scores. I don't care if you slept in or went to the doctor. You get three. That should cover life. Don't make me play detective.

Selecting Readings and Reading Length: Readings should be no more than about 7 pages or 3 arguments. Think about what you can cover in a class period. Can you cover more than 3 core arguments? My experience is, no. Not with any depth or discussion. So, why assign what can't be adequately covered?

Also, if the reading is longer than 7 pages, students won't read it. Remember they have 4 other classes. If every class assigns 7 pages per class that's 50 pages of reading for each class meeting. That's 150 pages per week if instructors only do lectures and no activities. That's just not going to happen. Don't set your students up for failure. I try to assign about 5 pages if it's dense and 7 if it's from a non-academic source.

Benefits to you: There's a happy upside to this. You only have to prep for 5 pages twice a week rather than for a chapter three times a week. You're welcome.

How to Design your Quiz: 
The quiz should not be difficult. Basically, you want it so that the average student will get 3/5 or 4/5 if they read the article. We're not trying to trick the students here. We're only giving them a small reward to do the reading. We are telling them, "I value you reading this article. Here's a cookie for doing it."

Question 1 should always be: "What is the main point the author of the article is trying to convince us of?" The other questions should involve core sub-arguments or sub-conclusions. Stuff like,

The author gives 3 reasons in support of X. Which of the following is not one of the reasons.

You can also do some obvious application questions: E.g., "What would the author say about the following case:[...]"

The point is that we need to reward them for the things we want them to do. We want them to read, so we reward reading. If they read the article, the quiz should be fairly easy.

Variation: Marcus Schultz-Bergin has a nice variation. Students are allowed to have notes open for the quizzes. This incentivizes them to be active readers and take notes while doing the readings. I'm contemplating using this myself.

Getting Students To Pay More Attention in Lecture.
Set the online quiz so that when they get a question wrong, the correct answer isn't revealed and build your lecture around the quiz questions. (And obviously ban cell phones once they've taken the quiz)
Most students will have gotten at least one question wrong on the quiz. They want to know what the right answer is. So, build your lecture (in part) around the questions on the quiz. If you built the quiz out of the key arguments and points, this should be fairly simple to do. Now, your students are looking for particular information throughout the lecture.

I know what you're thinking. But why should they care about learning the right answer if they've already lost the points for it on the quiz? Let me explain:

Quiz Redo: At the end of the week, students have the option of retaking their quizzes. The score of the first attempt and second attempt are averaged. This way, they're still penalized if they didn't do the reading the first time around but are incentivized because they can still improve. Also, the more poorly they did the first time around, the more incentive there is for them to listen and take notes during lecture. They are rewarded for paying attention and improving. Message to students: I value you improving and learning--See! Here's a cookie for doing it. Motivation problem solved. Learning is happening.

Details: Redo quizzes are open from Friday after class until Saturday evening. I don't allow the redo immediately after the first attempt because I want them to have to go back to their notes. This will better reinforce the information. I don't extend the redo time to Sunday because I don't want the quiz -taking to interfere with them doing the reading due Monday. Boom.

Activity Fridays
Imagine a world where you're told to read an article then you receive a lecture on it, never to hear about it again until six weeks later when you're asked to explain some ideas in that article from memory. Add to this that you have to do this for six weeks of other readings in five other classes. Gee, I wonder why students don't do well on exams. Then, a few months later at the end of the semester you're asked to write a paper that incorporates many of the ideas from the entire course. Your teacher is shocked and dismayed when you can't do this.


What do we want our students to do? Learn the content. Learn to apply the content. Learn the theoretical implications and trade-offs of the content. When in the above process did they do this? Answer: Nowhere.

Let's fix it.

Every Friday (or every 3rd period--you choose) I have students do activity sheets. The sheets are divided into three sections. As you might have surmised, the sections are: Basic content questions, application questions, and theory-level questions. Most groups create a group google-doc. The assignments are short enough that the average group should be able to do 80% of it in class and the fastest group should finish in class. However, it isn't due until the day after the next lecture (i.e., Tuesday at 5pm in my case). This way there's ample opportunity to polish. And--let's be honest--I ain't grading it until the following weekend anyway.

Fact: Most group-work is a dismal failure if it's assigned for outside class time. This has remedies but they're fairly involved so I'll set them aside. Besides, the purpose of the weekly group activities is fairly circumscribed so having them do most of it in-class works best. Also, I'm there to give immediate feedback and assistance if they're struggling or just need confirmation.

Here's a problem of group work that arises regardless of whether it's in or out of class: Some members do better work than others. If members are graded as a group, the students who did better work get penalized by the bad work. This is compounded by another problem: Students usually parcel out work for group work, so no one learns the content from questions they didn't do.

Solution:
Tell the students, "I will pick 3 (or 4) questions at random to grade. Your score on those questions determines the group's grade on the assignment." Now, rather than each student working on their own questions and failing to learn/reinforce the content of the other questions, students have an incentive to at least work in pairs and check each other's work. [Previously I graded the entire assignment but I'm going to switch to this method]. Because they don't know which questions I'm going to grade, students are motivated to do well on all the questions and check each others work.

Benefit to YOU: Not only do you avoid grading a bunch of individual assignments but you also don't have to grade all of each group assignment. Onerous grading averted. You're welcome.

How to Get Students to Perform Better on Exams
Ok, I've just explained how we achieve and reinforce the various levels of learning. What about this exam stuff? The midterm exam will be composed exclusively of questions from group-work assignments. Now they have yet another incentive to carefully answer the questions.

Benefits To You: You don't have to come up with all-new midterm questions. Just select from the ones you already have on the sheets. Besides, everything you want them to be able to know and do should be on those sheets.

But here's the biggest benefit. In the old school method you had to prepare 3 lectures a week. That's a pretty big time suck especially if it's a class you haven't taught before. Now you're only prepping 2 lectures a week. You just cut prep time by 1/3 over the course of a semester.

I know what you're probably thinking. "Well, yeah, it's one less lecture but I still have to make the activity sheet." Yes and no. You do have to make it BUT you build the activity sheet as you're making each lecture--not on a separate occasion. So, on Sunday night as I'm making my Monday lecture, I'm also writing down questions I want the students to internalize based on the lecture I'm going to give!  For example, if in my lecture I'm covering Aristotle's definition of happiness then I go into the activity sheet and guess what question I write? [Whisper: Explain Aristotle's definition of happiness]. Then in the application section I'll present a case and ask whether Aristotle would consider such and such a person to be happy. Next comes theory...I might ask them to contrast or defend Aristotle's view against a hedonist view we covered earlier in the course. I repeat this process for Wednesday's lecture.

When Friday comes around, the activity sheet is already complete because I made it Sunday and Tuesday night as I was building my lectures. Now instead of making yet another lecture and grinding yourself into exhaustion, you can drink yourself into a stupor or do whatever it is you like to do on your nights off. You're welcome.

More on Exams
I think of exams as a test of my teaching, not of my students' learning. If I structured the course well, lectured clearly, allowed them to interact with the content sufficiently and in various ways, gave the right incentives, gave them a reason to care about the content, then the majority of my students should do well. If they didn't do well then I need to change some things. If I've done my job, the class average should be a B.

If that sounds high consider what's happened in the course leading up to the midterm:

• They've actually read the material, 
• They've gotten feedback on how well they initially understood the material (i.e., the first quiz),
• They've had a lecture on the material where any misunderstandings can be clarified,
• They've worked collaboratively with others to ensure that they can comprehend the content, apply it, and understand the relative theoretical implications and trade-offs,
• They've had an opportunity to improve on their previous quiz and are rewarded for it; i.e., via the redo quiz they reread their notes and group assignment answers and they get more feedback on how well they understand it.
• They get feedback on their group work.
• They get a review session where they can clarify any uncertainties. (i.e., they interact with the content for the SEVENTH time.)
• They study the assignment sheets in preparation for the exam (that's EIGHT times).
• They take the exam.

Again, if I did my job right, a student would have to make a concerted effort to get anything below a B. They would have to willfully ignore everything that's happened in and out of class. (Yes, there are such students but they are a small minority.)

Here I have given you the blueprint for student success without overburdening yourself with grading and class-prep time.

My Studints Dont Rite Good and Reading There Riting Makes me Want to Stab my Eyeballs with Hot Needles

I'll elaborate more in a future post but here are the basics:

After the midterm I assign and grade a short paper (3 pages) with extensive feedback. This gives students a feel for how I grade and what I'm looking for (and of course I've also told them this when I assigned the paper). For all other papers there is peer editing (I only assign one more longer one). For their term paper I give them about 10 days to write the best paper they can. It's due 10 days to two weeks before the final version is due. I emphasize that it must be what they consider to be a finished version--not something thrown together the night before. I have them bring two printed copies and we do an in-class peer editing session.

Each student must edit 2 papers (meaning each paper is edited by two students). We do it on a Friday. Half of the time is allotted for each paper and what isn't finished is due on Monday in class. I've created a fairly extensive peer editing sheet. It should take around an hour for each paper. Their peer editing is worth 20% of their final paper grade (10% for each one) AND 20% of their final paper grade is how well they respond to each peer reviewer. Since they have motivation to edit each other well AND respond to the suggestions, the results have been great. What gets rewarded gets done. You can't tell them to invest a lot of time and effort into doing a peer edit then not count it for anything. Similarly, we can't expect them to respond to peer editing if there's no value in it for them. "I think it's valuable to respond to your peer editors. See! Here's a cookie for doing it."

Again, we want student to get better at writing. But you can't get better at anything unless you have a chance to learn from and correct your mistakes. If you simply ask students to turn in a paper at the end of the semester, when did they get to learn how to write better? Where was the opportunity for improvement? Most students just look at the final grade on their paper and that's as far as it goes. Responding to peer editing and rewarding it provides the opportunity to improve, and that's what we should be aiming for.

Also, as most of us know, weaknesses in own reasoning and writing are often invisible to ourselves--otherwise we wouldn't have made them public in the first place! Peer review allows us to recognize our own errors and weaknesses in the work of others. When we come back to our own work, we are better able see the once-invisible problems. Through peer editing we become better writers because we learn to edit ourselves.

If you'd like, here's a copy of my peer editing sheet to use or modify.

An Objection
I want to address just one of many possible to criticisms of my method. By using carrots and sticks I'm not teaching them the intrinsic value of reading, learning, knowledge, writing, etc... I'm only reinforcing what they've long been taught: That education and learning are primarily valued for extrinsic reasons.

This is where YOU come. The passion you bring to the classroom, the ways in which you tie the content to their lives and concerns, the encouragement you give, and the readings you select will all contribute to this end. Not all material speaks for itself for everyone.

I think it's unrealistic to think that most students will be purely intrinsically motivated from the start--especially if the course isn't an elective for them. But there's no reason why we can't have both intrinsic and extrinsic motivations. Some might only respond to the extrinsic motives. Fine. But my experience has been that if we can just get them to do the reading, students will start to appreciate the intrinsic value of the content. If I need to use extrinsic motives to get there, so be it.

This is not to say there aren't legitimate worries of extrinsic motives for education crowding out or corrupting the intrinsic ones (See: Michael Sandel). Maybe I'm just being pragmatic. Wait. No. I'm not. I'm also an idealist. I really do believe that if I can just get a little bit of engagement, most students will come to see the intrinsic value too. Anyone in this line of work has to believe that what they're teaching has intrinsic value. Also, it's possible that students do sometimes see the intrinsic value of a class but for pragmatic reasons, they don't do the work. They've got a life and concerns outside of school, just like us.

We must fight fire with fire--or pragmatism with pragmatism. If it's pragmatic reasons (e.g., a part-time job) that prevent a student from acting on what they perceive as the intrinsic worth of doing a reading or thinking about their paper then I'm going to give them pragmatic reasons to counter those obstructing pragmatic reasons: "Federalist #10 is one of the greatest American political documents you'll ever read. It'll change how you think about democracy and about your government."

Still not enough to get you to read it? Ok.

"Here's a cookie for reading it."

Hey, teach! This is good shit!