The data is what it is...

Every now and then, you think something should be pretty obviously true, but it turns out to be false... or at least not entirely true. Consider the following supposition:

Students who take their remedial math class and first college-level math classes in consecutive semesters are more likely to pass the college-level class than those who take a break between the two.

On the face of it, it's seems arguably plausible and pretty simple. And when you look at the numbers, it turns out to be true. From the data available, we found that 52% of students pass when there is a break in between and 62% of students pass when there isn't. This is a 10% difference and a relative 19% increase.

However, when you take a closer look at the data, a surprising trend arises. For the weaker students (B- to C- students), the pass rate is approximately the same regardless of whether they took the classes sequentially! The data set got a bit small for this part of the analysis, with less than 100 students, but the signs point to something lurking underneath the surface that has not yet been identified.

The mystery to understand why this is the case begins...

Looking ahead

I went to my first meeting yesterday. The discussion addressed something that I was not really aware of as an "outsider" to the department. One of the major problems of the camus is that students are not exiting with as high a level of basic mathematical competency as they would like. I didn't say much during the meeting, but there were a few things which seemed very reasonable approaches to the problem

• No gaps between math classes: Many students (for whatever reason) choose to take several semesters off between their math classes. Intuitively, this seems entirely counter-productive because math, like any other skill/knowledge, gets lost with lack of use. It seems unclear whether it is possible to mandate students to take the courses in consecutive semesters, but it is something to be pushed strongly.
  


• Tracking student data: Apparently, the Nevada System of Higher Education is using a very very old program to track their students (an upgrade is in the process, but it's a number of years off before completion). This means that while there is lots of anecdotal information to support various positions, it is hard to produce the hard evidence required to back it up. Apparently, some of the faculty figured out how to access the information and get the data they want out of it, so there's an effort to get this information and process it so see if the data supports the claims.
  


• Online class limitations: Apparently, there is an effort to make a lot of classes available online. There's even an effort to have entirely online degrees. I have some misgivings about online degrees (quality control, making sure that the name on the application actually corresponds to the person doing the work), but I can see the value of it (in principle). However, the degrees that some people are trying to push include things like physics, biology, and chemistry, which is absolutely absurd. These lab component of the physical sciences is such a large part of the degree that I cannot imagine that the degree would be given much credibility if it were earned entirely online!
  

I guess if I had thought about it for a while that I could have surmised that there would be conversations of this sort in a growing campus. But actually sitting in on such a meeting and hearing the discussion really drives home the feeling of urgency to "get it right" and to provide a quality product for the students, and to put students in the best possible position to succeed. (Of course, success rests on their shoulders, not ours -- if they don't put in the work, they shouldn't earn the degree.)

First class.... sort of.

I filled in for Jason yesterday in his precalculus class. I didn't do much lecturing, but instead spent the whole time answering questions. As a result, it felt a whole lot like teaching section all over again.

I didn't do anything different from teaching sections back at UCSD. I asked them what question from the homework they had problems with, and then I proceeded to explain the concept and solution to them. However, I did notice a distinctive "desire" to learn, mostly from the students who seemed to be non-traditional (I originally typed "older", but I feel that it might be received as derogative to some).

There was one student in particular who struggled with something involving absolute values, but could not find the right question to ask to solve her difficulty. I had to ask her a couple times to try to rephrase her question because I couldn't figure out what was tripping her up. But she politely and patiently persisted in trying to wrap her mind around it until she was finally able to put it together. I've seen that sort of thing happen before at UCSD, but I also remember the student simply say "never mind" after struggling with it for a couple minutes.

I was a bit amused by her response after she put it all together: "Thanks for humoring me." That's a new one. I hope to see more of this from my evening classes this Fall.

Learning Styles

I recently got my login information for the Blackboard Learning System at NSC and have been playing around with the different links that I found there. One of the links is the Index of Learn Styles Questionnaire. I thought it would be interesting to take the quiz and see what the results say about me.

Results for: AW

      ACT                              X                    REF
          11  9   7   5   3   1   1   3   5   7   9   11
                             <-- -->

     SEN                      X                            INT
          11  9   7   5   3   1   1   3   5   7   9   11
                             <-- -->

     VIS          X                                        VRB
          11  9   7   5   3   1   1   3   5   7   9   11
                             <-- -->

     SEQ      X                                            GLO
          11  9   7   5   3   1   1   3   5   7   9   11
                             <-- -->

The test took only about 5 minutes to complete. It's 44 questions long and I probably spent about 5-7 seconds thinking about each one. How do you read the results? You can read about the four categories on the Learning Styles and Strategies page.

I'm not very surprised by the first one. I tend to learn well in all sorts of settings and I don't really believe I'm strongly biased one way or the other. The second is also a balance that doesn't surprise me, either.

The last two are clearly dramatically skewed. I had initially thought that I would be balanced between visual and verbal, since I tend to be able to hold a lot of auditory information in my head, but after some reflecting I found that the questionnaire is probably right. Memorizing information is not the same as learning. So while I can hold audio in short term memory, it's not information that is being processed in any way. When students ask me questions in section, I tend to have to write down what they say and stare at it before I can see what they're asking.

Seeing that I'm a sequential learner is unsurprising. I've always been better at the "nuts and bolts" operations than trying to see the everything at once. I think this is part of why students tend to like my teaching style: I often break things down into small steps (or sometimes decision trees).

I'm greatly amused by this sort of thing. Perhaps this is an indication that I should really start looking into Math Ed research or at least SoTL research as a scholarly pursuit.

Spring 2007 Math 20D Evaluation Results

If you've looked at my poker blog, then you would know that I like record-keeping and data. I've compiled the results from my TA Evaulations and put them in a this document for future reference.

What must I get on the final to earn an A?

I bumped into a professor in the math department the other day on the way to the math department party and during the course of the conversation he shared the following little anecdote that I think is worth sharing:

Why is it that students ask, "How well must I do on the final to earn ...?" If you them that they must get 80% on the final to earn a B, are they somehow more capable of getting an 80%?

Non-rhetorically, why do students ask this absurd question? I don't know, but I think it's all an illusion and I dislike it. Instead of the "do the best you can" mentality I hope for students to have, the exam is seen as a game of grades. Yes, the assessment leads to a grade, but the grade is the afterthought to the educational value of the content being taught. The right question is "What can I learn?" not "What must I do?"

Thinking about this question has made me contemplate the type of response I should have to students who ask that question. I have gone through a few which I've listed here along with some commentary:

• "Your best chance of getting the best possible grade is to get a perfect score." - I'm not happy with this because it brings the grade into focus, and not the act of learning.
• "Just do the best you can." - While this is better, I don't think it demonstrates to the students that I care. Instead, this sort of sidesteps the question and doesn't give a helpful response (of course they will do the best they can).
• "Does it matter?" - It's getting closer. A question puts the ball back in their court to try to justify why they care. I'm not quite happy with it because I know the answer. It matters because they care about their grade.
  
• "Would it change anything if I told you?" - This is the best answer I have so far. It puts the responsibility on them to look at their question in a different light. I'm not denying the importance that they have put on their grade (as misplaced as I believe it is), but I am leading them to realize that this question is ultimately unhelpful to them.
  

I won't have to face this question for about 5-6 months, so I'll probably put these thoughts on the back-burner for now.

Evaluations

I've tried to do TA evaluations for every single class. I've tried a few different types, but the majority of them were this form from the UCSD Center for Teaching Development. I think it's a pretty good form, but now that I'm starting to think even more about teaching than I had before, I think it's a mistake to use that form over and over again. It is a good place to start, but I think that evaluations need to be customized to the goals of the class.

After all, in any class you teach, you set out with specific goals in mind. The evaluation should ask whether those goals were accomplished. As I read through the form that I had given to my students over and over again, I felt that I should try to make some adjustments to reflect my personal teaching goals. I wish I had thought to do this more than a couple days before I needed to have them filled out, but I still think I have a decent first level modification. You can download it here.

I haven't actually read their responses because I haven't graded their exams yet. This is something that I do more for myself than for them. As a matter of avoiding the appearance of impropriety, I don't want to give students the chance to perceive any undue influence on their grades as a result of their comments. I doubt this makes any difference in their responses, but I know for a fact that I can wait a week to find out what they said.

I'm just going to go from top to bottom and describe why I'm interested in their responses to various questions.

I borrowed the first couple questions from the CTD form, but I think they're wasted questions. How often students attend lecture is a reflection of the professor and not me. Also, by the end of the quarter, the students who are completing the evaluation are almost uniformly those who least frequently.

1) Aaron is easy to understand (both in speaking and writing).

There seems to be a recurring theme when people find out that I'm going to teach math. "At least you speak English." For some reason, it seems like lots of people have had bad experiences with TAs who struggle with the English language. However, my purpose for asking this is because I know I have a tendency to move really quickly through material. I am consciously aware when I start thinking faster than I can talk and when I start stumbling over my words. I also know my handwriting isn't always neat. It's generally legible, but I still want to keep track of these things as I continue to teach so that I don't become lazy about it. In retrospect, this should have been a 1a) and 1b) type question to isolate the the two parts.

2) Aaron has encouraged student participation.

There's a subtlety here that I think most students will catch. The question is whether I encourage student participation, not how successful I am at it. I've had some classes that are basically unresponsive no matter how I try. However, I think that students can see when you try to reach out to them and when they fail to respond.

3) Aaron has made the class more understandable.

This is an important marker for me. Students can often tell the difference between understanding something (or at least being in an illusion of understanding) and when they are going through rote motions.

4) Aaron has taught me how to think about the problems instead of just giving me the answers.

Most of my section time is filled with giving away answers. 90% of the time is spent doing some part of a homework problem. I want to know if students feel like I'm teaching them instead of just doing homework on my own. A more interesting question might be the following: I learn to think through problems as Aaron presents them instead of just copying the solution from the board.

5) Aaron has been helpful and accessible during office hours and via email.

This is a pretty standard availability question that could have been broken into two parts like #1.

6) Aaron managed section efficiently and effectively.

I believe that this is an area of strength in my teaching. I am very time conscious and I try hard not to go beyond the 50 minute section time. I also try to get to as many questions as possible in the allotted time. In my mind, this is more of a personal check-up to make sure that my perception doesn't deviate too far from the students'.

7) If given the chance, I would choose Aaron as my TA for my next math class.

This is the big picture question. Have I done well enough for them to want me to do it again? This allows the students to boil everything down and think through what they think is important and tell me if I'm meeting those educational needs. When it occurred to me to ask this question, it made me wonder why it wasn't on any of the other forms from the CTD.

I neglected to ask a couple things on this form:

• Aaron treated the students respectfully and ethically. (Taken from my Standard Syllabus)
• What could Aaron have done better? (An open-ended question to allow students space to comment on whatever they want)

I like how the self-designed evaluation feels. I can see some risk in asking questions that simply seek to affirm myself as a teacher, so I do need to find ways to ask questions that can help students to verbalize things they would like to see changed. This is more difficult because it's asking them to see what's missing. I think with more than a couple days' notice, I might be able to come up with questions that do that. The next evaluation won't happen until December, so I hope I don't forget. (That's why I have this blog!)