Reworking the Grading System

Chapter 7 of What the Best College Teachers Do talks about evaluating students. While I can't recall any specific passage or thought from the book that got me thinking this way, it did prompt me to think more about grading systems.

What does it mean for a student to have a problem "80% correct"? If I cannot answer this question, then it makes no sense for me have some conclusion that sounds like "therefore, 80% is an B- in my class." This is one of the reasons I am thinking of grading problems out of 5 points (see the last part of this post). I want to have a grading system that is simple enough to be consistent, but diverse enough that there are enough strata to have an accurate gauge on students in the class.

When I look at student scores right now, I see a spreadsheet that only has the final scores on tests. That means in a hypothetical 4 problem exam, I could not tell the difference between an 80-80-80-80 student and a 100-100-100-20 student. I would feel that the 80 student is probably a stronger student than then 100 student because the 80 student has shown that he has an understanding of all the topics, but the 100 student has not developed the same breadth of knowledge. This is also an indication of me not really knowing what it means to be 80% correct. Alternatively, it could also be that the 100 student just made some small error at the beginning of the last problem and that ruined the rest of his otherwise perfect work.

My current thought is data intensive, but sounds like something I might be willing to do with three small classes. It might be beneficial to keep a record of all the grades on the individual problems (homework and exams). This has multiple benefits:

1. I would be able to see overall trends in the class. If there are specific sections that are giving students more difficulty than others, it will be immediately apparent and give me a chance to cover that material more carefully.
2. It can get students away from thinking about their understanding in terms of percentages. I could assign grades based on the relative numbers of points they got on their problems, and not the sum of their scores (this would also require me to use a grading system of 4-5 points for every problem so that it is a consistent measure).
   
3. It may also provide good feedback for students if this was presented to them in a reasonable manner. It would have to be organized in a nice way and not just a list of numbers.
4. It will also provide historical data to recognize (for example) that most students really struggle with this particular topic, this problem is extra tricky, or other observations that could potentially slip through unnoticed.

As always, the devil is in the details because I have no idea at this point how such a system could be presented to a class in a way that would make sense to them. I need to think about this a little bit more.

First Day of Class Quiz

I've been toying around with an idea to overcome some of the hurdles discussed in this post. In particular, I want to change students' perception of what math is really about and the negative perception that they either are not good at math or cannot become good at math (obviously, I'm not talking about all students, since some don't have this problem).

My idea is to open up the class with a quiz. It's not so much a quiz as a self-assessment. I want to get them to start thinking differently from the beginning. My mind is geared right now towards my Elementary and Intermediate Algebra class (Math 097, Section 3). Here are some of the questions I want to ask (besides name, email, and those things):

1. Why are you taking this class?
2. What are the necessary skills required to be good at math?
   
3. What do you hope to gain from taking this class?
   
4. On a scale of 1-5 with 5 being "highly proficient", rate your mathematical ability.

I want to do a last day of class something where I give this back to them and have them evaluate the class. This is still a work in progress.

Stubborn Students

Here is a brief passage from Ken Bain's What the Best College Teachers Do (Chapter 2, opening pages):

In the early 1980s, two physicists at Arizona State University wanted to know whether a typical introductory physics course... changed the way the students thought about motion...

Did the couse change student thinking? Not really... They had memorized formulae and learned to plug the right numbers inoto them, but they did not change their basic conceptions...

[The professors] wanted to probe this disturbing result a little further. They conducted individual interviews with some of the people who continued to reject Newton's perspectives to see if they could dissuade them from their misguided assumptions. During those interviews, they asked the students questions about some elementary motion problems, questions that required them to rely on their theories about motion to predict what would happen in a simple physics experiment. The students made their projections, and then the researchers performed the experiment in front of them so they could see whether they got it right. Obviously, those who relied on inadequate theories about motion had faulty predictions. At that point, the physicists asked the students to explain the discrepancy between their ideas and the experiment.

What they heard astonished them: many of the students still refused to give up their mistaken ideas about motion. Instead, they argued that the experiment they had just witnessed did not exactly apply to the law of motion in question; it was a special case, or it didn't quite fit the mistaken theory or law that they held as true. "As a rule," [the professors] wrote, "students held firm to mistaken beliefs even when confronted with phenomena that contradicted those beliefs." If the researchers pointed out a contradiction or the students recognized one, "they tended at first not to question their own beliefs, but to argue that the observed instance was governed by some other law or principle and the principle they were using applied to a slightly different case." The students performed all kinds of mental gymnastics to avoid confronting and revising the fundamental underlying principles that guided their understanding of the physical universe. Perhaps most disturbing, some of these students had received high grades in the class.

I earned a B.A. in physics as an undergrad at UCSB. (I didn't earn a B.S. because I didn't take the physics labs; they were too time consuming and I wasn't intending to continue in physics.) I was fortunate enough to be in a specialized program (College of Creative Studies) where we had a very good teacher for our lower-division physics classes. He taught us in ways that forced us to think about how we thought about the subject by assigning very difficult problem sets. What was difficult about them? It wasn't only computationally difficult, but he asked us to interpret the meaning of the results. This forced us to reconcile our results with reality and expanded our ability to think physically.

Here are a couple examples of things he would do:

1. In multi-body problems, we would see what would happen to the results as one of the masses becomes infinitely big or small. Did our new results match the expectations? Why or why not?
2. Some problems resulted in two solutions. Did they both have a physical interpretation?

I would like to find ways to do this with math. I don't know how it can be done, but I expect that it can be done. The details will have to wait for a specific class, or even a specific topic or problem, because I think it's impossible to talk about real life in the abstract. It can only be done in the context of things that are actually happening.

The book continues to discuss the views of the successful teachers with respect to the development of knowledge. I'll give the bullet points here and have you buy/read the book for yourself if you want to find out more:

1. Knowledge is constructed, not received
2. Mental models change slowly
3. Questions are crucial
4. Caring is crucial

The biggest hurdle for the students who are not mathematically inclined (most of them) is that they come in with the presumption of what math is and that they aren't very good at it. They don't really focus on the processes, but the end result. Why? This is how they are trained to think about math in school from the beginning. (Read this post for more on this topic.) The biggest hurdle for me as a teacher is trying to turn away the negative light on the subject. I have some ideas, but I won't know if they will work until I get the chance to try.

The importance of thinking

I was up late talking with a friend the other night. He lived with me last year, but spent this year in China teaching English and computer skills in a rural town somewhere. We had a nice conversation about a number of topics, and one in particular was relevant to this blog.

It turns out from our collective experience that most undergraduate students don't know how to think about math at all (ourselves included). This doesn't mean that they are incompetent or stupid, it's just that they have not ever developed the skills of critical thinking and self-reflection.

Before talking about math, let me point out a specific example in real life. Over by the Mandeville center, down where the art people have a space to do their thing, there used to be a painting on the ground (someone painted over it). It was a picture of what I've always supposed was Mary and baby Jesus (woman, child, halo). Next to the figure was painted "Do not push your beliefs on others" (or something like that). I am not sure whether the person who painted that appreciates the irony. As a personal philosophy ("I will not try to force you to believe what I believe"), it works just fine. However, it cannot possibly be given as an instruction to someone else in an internally self-consistent manner ("You should not impose your beliefs on me" -- by making this statement, the speaker is trying to impose HIS beliefs on the listener, thus doing what he says one should not do). A lack of critical analysis leads to nonsense.

The same internal inconsistency exists with bland, naive relativistic statements such as "all religions are the same" (different religions have different claims to truth, and they are obviously incompatible) and "you should accept everybody" (the person doesn't accept you because you don't accept someone, but then that person is himself not accepting of everyone). Most of the people I know who talk that way haven't ever really spent time thinking and reflecting on their system of beliefs -- it's much like how Christians who never study the Bible come to false conclusions about how God "should" behave (really, how God "does" behave).

But what does this have to do with math? I'll grant that philosophical reasoning is a bit stickier than mathematical reasoning. At least with mathematical reasoning, we (the math community) have a set of mostly agreed upon fundamental beliefs (axioms) upon which we build mathematical structures. There also isn't a whole lot of room for "personal interpretation" when it comes down to the formalism of mathematics (there is actually room for personal interpretation when it comes to trying to understand math... but that's a different story).

Where was I? Oh yeah, what does this have to do with math? A lack of critical analysis leads to nonsense. The nature of mathematics for a lot of students is that you find some numbers, or a formula, and plug stuff in, and move some terms around, and get some answer. It doesn't matter where the problem came from and it doesn't matter how silly the answer is. It doesn't matter that it's a word problem computing the terminal velocity of a falling object, and that the final answer they got is "t = 10 minutes." The problems have no intrinsic meaning to the students, so their answers often have no intrinsic meaning.

Here are a few examples of relatively simple mathematical processes that a lot of people know they're supposed to do, but don't actually know why it's the right thing to do (other than they were told to do it that way):

1. Why do we "carry the 1" in addition? (Even more fundamentally, why do we start on the right side when we add? -- This one isn't about being right or wrong)
   
2. Long division: Divide, subtract, multiply, bring down, repeat... What does this process actually do?
   
3. x^n * x^m = x^(n+m) -- If you're not familiar with typing math, x^n is x to the nth power.
   

The lack of understanding of these operations is the result of how math is taught. My memory of learning math in the California public school system goes something like this:

• Kindergarten - Learn to count
• 1st Grade - Learn to add and subtract
• 2nd Grade - Learn to add and subtract in multiple columns
  
• 3rd Grade - Learn to multiply, introduction to fractions
  
• 4th Grade - Learn to multiply in columns and do long division
  
• 5th Grade - I don't even remember what I was supposed to learn...
  

The point here is that math was taught as a process and a bunch of rules that you memorize and reproduce. There were some parts where math's intrinsic value was highlighted. For example, I can remember "clock arithmetic", which was a low-level introduction to modular arithmetic, but it still played out like another set of rules to memorize.

I was fortunate that my school and my teachers were willing to put in extra effort for the bright students. There was a weekly thing in the library for the more proficient math students (I remember learning about exponential growth there by the chessboard and grains of wheat problem).

My fourth grade teacher borrowed books from the library to keep me occupied while she taught arithmetic to the rest of the class (she got me books on finance, but since I didn't have a good conception of money at the time, the real value didn't sink in until much later... for example, if I understood no-interest government student aid loans and all that stuff, I would have realized that when I started my undergraduate studies, I could have taken out a maximal student loan and earned 5% interest in a CD somewhere and make free money off the government. Even at 3.5% APY on a $10,000 student loan, because I was in school for 9 years and the loan would be without interest would have given me an extra $5500 for free. Then taking the $5500 and continuing to earn 5% APY on it from age 27 to age 65 is an extra $35000 in retirement funds for doing essentially nothing. Of course, I probably could have qualified for more at the start, and probably could have gotten better than 5% return on average (I think the average market growth rate is around 8%), but I think this makes my point about the gap between being able to compute things and being able to critically analyze what the math is saying.

Who is to blame for the inability of students to think critically about their math? It's not the fault of any individual, but it's a faulty system. Teachers teach math this way because this is basically how they learned it. Students learn math this way because they have no other model for learning math. The students grow up, and this way of thinking is never corrected, so it propagates itself to the next generation.

Fortunately, I do believe change is in the works. There are a number of educators who are looking into the failing mathematics system, and while I'm not involved there, I do hope they find true solutions and not false ones like having teachers teach to standardized testing... that just reinforces the same problems.

MDTP San Diego Conference 2007

MDTP stands for "Mathematics Diagnostic Testing Project". I attended the conference without really knowing (or caring) what MDTP actually does. I went because I was interested in learning about teaching and expanding my ideas about teaching.

The first talk I attended was given by Jeff Rabin (UCSD). I've met him a couple times on an inter-personal level and I took his course mathematical methods course my first year here. He opened by introducing the following problem:

Give an example of two triangles that have 5 congruent parts but are not congruent triangles.

He gave everyone about 10 minutes to work on it. I think I got it at around 7-8 minutes. More important than actually obtaining the answer was the intellectual process that one takes to get from the question to the answer. I can write down an answer and (if you're mathematically inclined) it will take mere seconds for you to conclude that it is a valid example. However, the point was to notice the thought patterns involved in working out a problem like this. There were two important features in play:

1. I did not know the answer ahead of time. I only knew that an answer existed. (Alternatively, the question could be phrased "If possible, give an example..." This would create an interesting situation in a classroom setting with more time for interactions.)
2. I did not know the specific tools required in order to obtain the result. However, the fact that we were talking about incongruent triangles gave enough of a hint as to where to start.

This is the essence of problem-solving. Finding and justifying answers to questions whose answers and method of solution were not previously known to the problem solver. I actually don't remember what else he had to say, but this was already useful information to me. I would like to keep this in mind as I plan problem sets for students in my classes. Of course, there is also the necessity for having exercises (straight-forward computations, questions where the method of solution is actually known in advance -- perhaps given in lecture at some point).

The plenary session was given by Guershon Harel. I met with him once when I was thinking about post-grad school jobs and ways to perhaps transition into math education instead of researching in pure math. He talked was titled "Thinking in Terms of Ways of Thinking." He talked specifically about DNR-based instruction (Click here to read a short paper on it), I would like to find some way of having math students (especially those not going into math) to evaluate themselves and how they think about things. I do like to emphasize that there are different skills in mathematics, which is something I talked about in the Teaching Statement I used when I was applying for jobs. My ideas were not nearly as refined and technical as his, which is perfectly fine by me.

"I don't know" is a good answer

In the summer of 2006, I had the opportunity to teach Math 10A (calculus for the non-technical students) through a fellowship from UCSD's Center for Teaching Development. You can see the syllabus here. I think the format is a little bit clunky, but I'm not very fluent with HTML and making the boxes that resize themselves properly was a big deal for me.

This post is going to highlight the "About Tests" section, which I've copied below:

Tests measure your ability to demonstrate your understanding of the course material.

I have an unusual stance when it comes to exams. You can earn up to 20% credit for admitting that you don't know what you're doing instead of haphazardly guessing at what you should be doing. I want to discourage the "shotgun" method of test taking; that is, I don't think you deserve credit for writing down a bunch of stuff and hoping that some part of it resembles something that might come close to the right answer. The tests attempt to measure how well you understand the material, not your ability to spew information on your paper. This does not apply to multiple choice questions. On the tests, there will a box to mark if you want to take the credit.

Similarly, you will earn credit on your exams for having a good presentation. While the answer is important, it is also important that you are able to demonstrate how you got to that answer. Math reads left to right, top to bottom, just like in English. (It helps to practice good presentation by doing this on your homeworks!) IfI you have questions about the clarity of your presentation, you are welcome to stop by during office hours and I will help you out.

The 20% credit for not randomly guessing was an idea I came up with as a graduate student while I was lamenting the terrible scribbles students left on their paper when they clearly had no idea what was going on. It frustrated me enough to make me want to give negative points. Of course, that's not an option. I don't think it's good to penalize students in that way.

But instead, I think it's appropriate to award students for academic honesty and integrity by giving them the chance to say "I don't know." In real life, I think "I don't know" is a perfectly legitimate answer, and is often the best one when it's true. Too many times I have seen people (myself included) get trapped in difficult situations because they didn't want to admit that they were not qualified to give an answer on the basis of lack of knowledge or experience.

There are a number of positive aspects to this idea:

1. As mentioned above, it rewards students who are able to give academically honest answers.
2. It encourages students to evaluate the quality of their work, something which seems to be conspicuously absent, especially among students who are less mathematically inclined.
3. It prevents students from being penalized inequitably for that one topic that they never quite understood that happened to be the one that showed up on the test.
4. It makes grading those problems much faster.
   

As I tried to implement this, I discovered a few problems which I will hopefully be able to address and clean up with some more experience:

1. The grading must be done in such a way that 20% is a meaningful enough amount to make it worth while for the students to consider it as an option. Many students felt that their wild guessing would get them more points. I can see a two possible solutions. I can increase the value from 20% to 40%, or I can change the grading so that it is harder to earn 20%. I'll have to experiment and see what happens.
2. Many students don't know how to interact with this option. Their entire academic lives, they have been taught *NOT* to leave questions blank and to always guess something because "perhaps you'll get partial credit." I think students need to be retrained to use this system to their advantage.
3. The grading must be consistent from problem to problem. It cannot be difficult to earn 20% on one problem, then a piece of cake to earn 20% on another one. I think this can be resolved by making all problems worth 5 points. With a narrower grading system, there is less room for fudging around with -1 for this mistake and -2 for that mistake. What is the difference between 14/20 and 15/20 on a particular problem, anyway?
   

I know my son can't read...

Here's a story from one of my high school teachers. I think it demonstrates how some people have a poor sense of what it means to get an education.

Every student must pass a semester of civics in order to graduate from high school. Because everyone must pass this class, it's not particularly hard. Not everyone will get an A in it, but everyone who works at it should be able to get out with a passing grade. There was a particular student who was failing this class. The teacher called up the student's mother to talk to her about what's going on, and to encourage her to encourage him to put in the effort so that he can pass.

Initially, the mother tried to do some negotiating with the teacher, but the teacher would not budge. He refused (on principle) to give a student who was clearly failing the class a passing grade. After a while, the mother got exasperated with the teacher's position and said, "I know my son can't read, but I want him to have a high school diploma."

To me, this is very sad. It's likely that the mother does not have much of an education herself based on this comment. It's likely that her son does not appreciate the education he has been getting because he still can't read, even though he's in high school. (That he got so far in the first place is a sad commentary on the state of education.)

A degree will only have meaning if it allows one to differentiate between those who are qualified and those who are not. It's a system that does draw a very clear line between the "haves" and "have nots," and some people don't like that. I'm not ignorant to the fact that social conditions have an effect on students and the levels of education they are able to attain. I'm in favor of outreach programs for low income students and other things to help them to navigate the educational system (especially higher education). However, "help" should never be turned into
a "giveaway." The students must still demonstrate that they have the knowledge and the skills appropriate for the degree. Otherwise, all you do is treat a symptom without providing any real help to cure the problem.

The Cheating International Student

This is one of the things I don't look forward to about teaching. There are students who put themselves in bad situations and then look to you to bail them out. Here's a story that happened recently in a class I was TAing.

Before I begin, I will say that I don't know the student personally (I don't even know his name) and I don't think there's really any information being put forth here that will compromise any sort of privacy or anything.

This student was suspected of cheating on the first exam. He didn't cheat during the exam, but he went home, changed some answers, then brought it back under the pretense of a regrade. The other TA felt as if something was funny, and made a photocopy of his second exam. Sure enough, he came back for regrades again, and was caught red-handed.

I would be completely oblivious to this if the student had not come to the professor's office to beg (literally) for mercy. I happened to be there to help him grade the final exam. He was an international student and kept asking for "forgiveness" (which is an entirely separate matter). He was clearly distraught by the prospect of being forced to leave and took a physical posture of submission by being on his knees. He cried a lot and kept repeating "forgive me, professor."

I found out later that this student was already on academic probation for poor grades, and that he was not doing well in his other classes. Failing this class would mean an automatic dismissal, but so would being caught cheating. The personal side of his story was that his parents worked very hard to send him out here, and there was an implied sense of shame if he were to return home because he was kicked out of school.

What do you do in this situation and what are the guiding principles?

As a Christian, my entire system of beliefs is based on grace and the delicate balance of mercy and justice:

Micah 6:8 -
He has showed you, O man, what is good.
And what does the LORD require of you?
To act justly and to love mercy
and to walk humbly with your God.

How do you act justly while simultaneously demonstrating a love for mercy in this situation? I talked with the other TA for a little bit about this student. He felt bad for the student and didn't want to come down hard on him, because people make bad choices and make mistakes, and they should not be held against them forever (being kicked out of school probably means he's going back to his own country, and he may never get the chance to come back). That sounds good because it sounds like mercy.

But what of justice? This student has already shown that he was borderline because he's already on academic probation. He was clearly caught cheating on an exam (the type of cheating that requires him to lie to the face of his TA). He did the crime, he doesn't come in with a clean record, and it is appropriate for him to receive some sort of punishment for this. That sounds good, because it fits exactly with how you're supposed to respond to a student caught cheating: Report it to the appropriate academic council and let the system do its thing.

The thing that got me the most about this whole thing was that the student never seemed repentant for what he did. All the time that he was saying "forgive me, professor" he was also trying to negotiate some sort of deal. He wanted to receive a D instead of an F (even though we had not even graded his exam, scoring 0 on the second midterm was almost certainly going to make him fail the class). He wanted to do something to avoid getting kicked out. To me, this weighs very heavily on the scales towards taking the hard line of justice.

Grace is undeserved favor, but mercy can depend on actions. People who "turn their lives around" are more deserving of mercy than those who continue to do wrong. In his approach to the professor, this student showed that he was not concerned with the actions that led to his situation, but the consequences of his actions. He was not interested on being on the side of truth, merely avoiding punishment.

In my mind, this student has already failed himself out of school. He didn't have just one bad quarter, or just one class where he was not doing well. He was on the road to not succeeding in school. Even if the professor gave him a D in this class, the student would not be much better off than he was before. He would still be just a borderline student, he is likely to cheat again (he's also likely to have cheated before). Allowing him to get away without recognizing the magnitude of his situation is not likely to benefit him at all. Perhaps he will finish and "earn" a degree. Or perhaps he'll fail out in the next quarter, or the next year. It's hard to say. If you send him away, he may understand that his actions have consequences, and then proceed to earn a degree in his own country (with integrity).

This reminds me of a story from one of my high school teachers, which I will post separately. Click this link to the story.

Things I want to post

It seems like a reasonable guess that I would not have started this blog if I didn't already have a few ideas of things I wanted to post about. However, I don't have the time or the energy to do this all at once, so I'm making a list of things to look forward to in the future.

1. Ken Bain's What the Best College Teachers Do.
2. The 20% credit for knowing you don't know experiment from Summer 2006. (DONE)
   
3. The MDTP San Diego Conference sponsored by the UCSD Math Department (3/29/2007) (DONE)
   
4. A rant about the unfortunate state of mathematical maturity in most of the students I have encountered as a TA at UCSD. (DONE)
   
5. My observation of the student caught cheating. (DONE)
   

What is it?

This is my third blog. The first, the Tired of Grinding Project was a record of my online poker playing as an opportunity for my own study of the game. I tried to make a parallel blog, The Hand History Project as a place for me to compile a bunch of hands that I played well to help me to reinforce the things I'm doing correctly and to help me to identify errors.

I have not posted on the first blog in nearly two weeks, but I also have not played any poker during that time. I still intend to keep that blog going. The hand history project is going to be a dud in my hands forever. I don't like how I started things off, and I don't intend to try again for a while. I think I would need to come up with an alternate system if I ever wanted to see anything result from there.

I like the idea of blogging. I like keeping records of things; it helps fight against selective memory and self-deception. I like forcing myself to express my thoughts in words; I tend not to believe that I can really understand something unless I can explain it well to someone else. I like the idea of being able to write down my thoughts at almost any time from almost anywhere (I almost always have internet access, but I'm not always near my own computer).

I am nearing the completion of my Ph.D. in math at UC San Diego and head out to Nevada State College, where I will begin a career as a college professor. I'm not afraid to admit that I'm looking to be a primarily teaching professor, not a research professor. After spending 5 years doing math as my primary focus, I have discovered that I don't like it enough to continue doing it indefinitely. However, having been a TA for 5 years, I can say that this is something that I am very certain will hold my attention.

Therefore, I am starting this blog. There are three major goals of this blog:

1. I want to record my thoughts about teaching. This includes reflections on things that I read, conferences I attend, or just general thoughts that float through my head. I will also occasionally have a new/experimental idea that I want to jot down somewhere for future reflection and potential implementation.
   
2. I want to record my teaching experiences. I taught a class less than a year ago, and I have already forgotten most of what happened during that class. I have some records that I can look up, but those will only tell me informational things about the class and very little about what I thought about things at that time. In other words, I can't get inside my head when I look back at those times. This blog will help me to remember.
3. I want to build up a good habit of professional reflection to become better at what I'm being paid to do.

That's all I have to say on my first post.