In praise of quals

One of the requirements for earning a PhD from MIT (or any university, as far as I know) is to pass a qualifying exam, colloquially known as “quals.” Quals vary by department, and the mechanical engineering quals are particularly notorious. I think this is partially due to their difficulty and partially due to their unfamiliarity. They are difficult: despite most students preparing for months, the first time quals pass rate is somewhere around 75%. And they are different: a large part of quals is oral exams, where you solve a problem on a blackboard in front of a panel of professors who can ask you questions to probe your approach and your understanding.

To me, the style of quals, which is so different than how students are traditionally tested, should be a reason to celebrate quals rather than dread them. Almost every engineering class I’ve taken has had tests, and all of them used the same format: I sat in a room, silently and without access to online resources, and wrote down answers to questions on a piece of paper (even when the course was about the use of computer software). I find this testing tradition absurdly contrived: when in your working life will you be given a problem like the ones asked in written exams without access to colleagues or the internet? Admittedly written tests are good at assessing how well you can answer test questions, but I think it’s a far stretch to claim that performance on a written test will accurately assess one’s competence at solving problems on the job.

The oral exams associated with quals, however, are much more closely aligned to how problems are solved out in the world. You are asked a question that doesn’t necessarily have a single, clear-cut answer, and you have to convince others that you have a strong enough understanding of the underlying principles for them to trust that your answer is reasonable. This is what I do when I discuss issues regarding experiments with my lab mates or when I field questions after giving a talk at a conference. Which is not to say that the usefulness of this skill is limited to academia – this exercise is central to technical consulting, for example, and I imagine that it shows up in every non-rote engineering job out there. I think it’s only because oral exams are so resource intensive (~3 profs * ~30 minutes per student) that it’s so uncommon for us to be assessed this way.

I am not arguing that quals are without issue (their high stakes nature makes them inherently stressful), but I would encourage any students taking quals to be grateful for at least some aspects of the opportunity. The novel testing environment is closely aligned with your best interests as a PhD student: by the end of the process, you’ll have convinced a cadre of MIT professors, through your direct interaction with them, that you are deserving of a PhD from MIT. The same could not be said had you simply completed another battery of written exams.

 

Thoughts on connectivism and cMOOCs

I was first exposed to connectivism last year in Chris Dede’s class on educational technology. That class was when I first learned (at least in terms of formal definitions) about constructivism, behaviorism, and cognitivism, so I was (and am) by no means an expert on learning theories. That being said, the idea of connectivism was definitely striking to me. A person’s productivity strongly depends on the connections they have (e.g., when they encounter an obstacle, do they know someone who has an easy solution, or would they always try to work through the obstacle them self?) so to me it makes sense that learning and education should be tied to those connections as well. From a purely practical perspective, people don’t work in a vacuum in “the real world,” so it doesn’t make sense to artificially create a vacuum for them to work and be assessed in at school.

But while I learned about connectivism last semester, whatever I learned about cMOOCs (if anything) didn’t seem to stick. Like most people exposed to some education journalism, at the beginning of MIT Massive “MOOC” was synonymous to me with what I now know is referred to as an xMOOC. Perhaps it was just that I needed a concrete example, but after exploring the ds106 page (a cMOOC on digital storytelling) I am very taken with the idea. It paints a picture of how learning should be: driven by student interests, supported by a community, and focused on outputs that students can be proud of (basically all the positive features listed in the DML Connected Learning report).

At some point, perhaps when I have more time (haha, the mantra of a PhD student) or I’m more interested in having digital storytelling be a focus of my personal development, I’d like to go through the open ds106 course. As tangential participation through MIT Massive, I just decided to try a couple of the daily creates, short suggestions for making something that are put together by the ds106 community. I enjoyed both of the exercises I tried. They were relatively quick (~30 minutes each), allowed me to exercise creativity, and had a nice product at the end. I guess most importantly, it was fun to go through the process of making them.

For TDC1385 I wrote calligraphy of some Drake lyrics:

Drake Calligraphy

When I first saw the prompt, I thought the idea of writing out Drake lyrics in calligraphy was hillarious: I like the juxtaposition of the old and new. I was confused that none of the submitted responses were in calligraphy, until I read the prompt more closely. Even though it called for an illustration of the meaning, I found the literal interpretation of the prompt too funny to pass up.

For TDC1358 I designed a custom road sign:

RoadSign

This was inspired by a run in I had with a Boston driver earlier in the day (I guess technically it was a Boston passenger, he wasn’t driving the car) who cursed at my friend and I for not crossing the street when we had a “don’t walk” signal. Sometimes it feels like whatever you do, there’s a Boston driver who thinks you’re an idiot for it.

Is there a limit to technology’s role in the classroom of the future?

This week in MIT Massive, we had an activity where we were tasked with designing a school which takes advantage of personalized learning (where in this case, personalized learning means software like Knewton or Khan Academy). All the groups in our class assumed that such software can save time used to the teach basic content, freeing up time to participate in deeper learning experiences (e.g., project based learning). None of us designed a school where the personalized learning was both the means and the end, where the core student experience was solely interacting with their computer. Given that we seemed to agree that computers weren’t better at teaching everything, Justin asked if we see a limit to the role that technology can play in the classroom moving into the future. I have some thoughts on the topic that I’ll detail below, but to give you fair warning, my answer to this question is probably more of an excuse to wax philosophical than it is an attempt to reveal any insight about the role of technology in education.

The short version of my answer is that I don’t think there is any limit to the role technology can play in the classroom. In the future, I think it’s possible that a student could get a better education wearing a VR headset connected to a consumer computer than they could get in the best classrooms in the world today. And the version of “better education” that I’m claiming is all inclusive: they would have stronger content knowledge, better critical thinking skills, be more motivated to continue learning, have a stronger internal locus of control, and score higher on any other metric you can come up with. I believe this for primarily two reasons: 1. human consciousness does not transcend any physical laws and 2. technology continues to march forward at an unrelenting rate.

For me, the first point is almost self-evident. While consciousness is incredibly complicated, and the current framework for understanding it is beyond me, I do know that consciousness arises from something physical: the connection of neurons in our brain. While it’s true that there are many, many neurons in a human brain (on the order of 100 billion), there are also billions of transistors in the CPU of the laptop I’m writing this post on. From a physics perspective, I don’t see why we couldn’t connect a bunch of computers together and have consciousness arise: physically the infrastructure is there, we just haven’t made the right connections yet. A more visceral manifestation of this idea is that if you walked away from the movie Her thinking that [spoiler alert] Theodore’s love for Samantha (the AI) was at least as real as his love of Catherine (his ex-wife) had been, you were convinced that there is nothing magic about humans that gives us a monopoly on consciousness.

The second point is also fairly self-evident in current society. Moore’s law, the doubling of computational power every two or so years, is a well known phenomenon. A popular statement of the the effect of Moore’s law is that the computer in your pocket today is more powerful (by many orders of magnitude) than the computers used to land a man on the moon. While Moore’s law has been slowing down in the past few years, it doesn’t change the fact that computers are still becoming more powerful, and we can’t imagine a limit to what they’d be able to do (or perhaps more accurately, the limit is effectively infinity compared to what they can do today). One of my personal favorite examples of how far computers have come is AlphaGo, the computer program which was able to beat one of the world’s best Go players earlier this year, a task which wasn’t expected to happen until 2025 by most experts.

If these two points sound like they’re adding up to the singularity, the idea that when computer intelligence surpasses human intelligence it will lead to an infinitely accelerating wave of technological progress, I guess it’s because in essence that is what I’m arguing. If we accept these two points, eventually we’ll have computers that can pass the Turing test which are cheaper than a smartphone is today. Every student could have a personal AI tutor with the capacity for thought of a human and instant access to essentially all of human knowledge. They could interact with this tutor in a virtual reality with a level of resolution indistinguishable from the real world. They could also work on projects in teams where their teammates are AI teachable agents, operating to provide their student with an ideal learning experience (both in terms of learning content and non-cognitive skills). I would not be surprised if this vision is frightening to many, and if that’s the case for you I would remind you that in this post-singularity scenario, interacting with an AI is just as rich and complex as interacting with another real live human being.

Of course, even if this future is possible (or inevitable), it doesn’t do much to inform how technology should be used in education today. Any skill that a computer can teach more effectively than a human can also be performed more effectively by a computer than a human, meaning that skill is not useful for a human to have in order to be a productive member of society. In this future, there would be no need for humans to do work and we could live idle lives full of whatever leisurely pursuits we found most interesting. So while there’s no limit to the role technology could play in the post-singularity classroom of the future, that lack of a limit is irrelevant to the classroom of today.

PolyCraft World: between self and peer-directed learning?

I missed the first session of MIT Massive because I was in Phoenix, AZ at the MRS 2016 Spring Meeting, a large materials science conference. I spent most of the conference listening to talks about solar energy and new results for multi-junction PV cells, but I was also introduced to one educational technology that really struck me. At MRS I met Dr. Walter Voit, the creator of PolyCraft World, which is a mod (essentially an extension of functionality) for Minecraft. Minecraft, for the uninitiated, is a wildly popular video game that lets players explore and build in a stylized, blocky world, which itself has been implemented in classrooms as part of structured curricula. Even outside the classroom, I would characterize Minecraft as a learning community: it has a huge community generated wiki and countless hours of YouTube videos recording the in-game creations of Minecraft players.  

PolyCraft World adds additional materials and tools to the base Minecraft game, with a focus on chemistry and engineering. In Minecraft, players use materials like wood, stone, and iron, while in Polycraft World they also have the option to work with dozens of types of plastic, among other materials. Part of the educational value comes with the fact that working with the new materials is scientifically accurate: a chemical processing plant in PolyCraft World performs stoichiometrically balanced chemical reactions.

One of the more trailblazing plans Dr. Voit has for PolyCraft World is to teach some sections of intro organic chemistry at UT Dallas entirely through the game. Discussing this, the part I was most excited about was how much richer assessment could be in the software (e.g., build a reactor that can produce the desired chemical product, with a minimal footprint) than in a traditional pen and paper exam (e.g., identify the correct ratios of products and reactants in the following chemical equation). Reflecting back, now I’m also curious about how PolyCraft World bridges the space between the peer-directed learning of Minecraft and self-directed learning (if successful) of a game that can teach college-level organic chemistry. In Dan Meyer’s argument against personalized learning, he warns against the potential loss of synchronous learning, but I wonder if a system like this could provide some of the best of both worlds: scaffolding and motivation when the student wants to work on their own, but a global community of potential collaborators to undertake a grand project with when the student wants to work with others. 

Why I made a personal website

One of the requirements of the course I’m enrolled in this semester, MIT Massive, is to create and maintain a blog. We were provided with a turnkey blog framework, but I decided to create a custom personal website instead and added this blog to it. Making a custom site was a lot more work than the default option (and I’ve barely added anything to it yet!), but I felt it was an important step in acting congruently with some beliefs I’ve developed about the current state of education related to assessment and GPAs.

The problems with GPAs

Anyone who has been through the U.S. education system is familiar with the grade point average, or GPA. Ostensibly, the GPA is a quantitative measure of students’ knowledge and competency in all of their academic pursuits. It is a convenient way to quickly compare different students, since it boils down their achievement to a single number. GPA is weighed heavily in university admissions decisions (for both undergraduate and post-graduate programs) as well as in hiring decisions at many companies, and I think this is the case primarily because using GPA is convenient.

I think the continued use of GPAs is a mistake and I’ve become increasingly convinced that 1. GPA doesn’t measure overall academic knowledge and that 2. using a single number to report academic achievement is hopelessly flawed.

What GPAs actually measure

First, if GPA doesn’t measure academic knowledge, what does it measure? Tautologically, a high GPA indicates that the student earned high grades, and this could have resulted from a number of reasons: they are smart, they are hardworking, they care deeply about their grades, they have deep academic content knowledge, they cheat on exams or their family has the resources to provide academic support such as tutoring, to name a few. While some of these reasons are aligned with the intended purpose of GPA, some are definitely not. Research suggests that GPA is most strongly correlated to self-discipline, which is undoubtedly a useful characteristic to measure and is important to success. That being said, because most academic environments are “artificial,” in the sense that there is a single correct answer a student is asked to find (a situation not often encountered in real, innovative work), some companies are finding that GPA has little to no correlation to job performance. I would also argue that even if GPA is a good measure of self-discipline, a metric which measured knowledge and competency more directly would be more valuable. In the current system, a lot of time, effort and money (on both the student’s side and the educator’s side) are spent awarding high GPAs and providing evidence of good work ethic without the student actually internalizing the material they are supposed to be learning. Anecdotally, I know this to be true because I took college freshman chemistry alongside classmates who passed AP chemistry (and therefore could have skipped the class had they wanted to), but still struggled with concepts that were covered in my intro high school chemistry class.

The fallacy of distilling student achievement into a single number

Second, why is trying to measure academic achievement with one number flawed? For a very long answer, you could refer to The End of Average by Todd Rose (a book which I thoroughly enjoyed and highly recommend), but even if we take “average” academic achievement as being an acceptable idea, the GPA still has problems. There are two main arguments one could make for the value of using GPA to judge students: that it is a useful tool for comparing the relative achievement of students or that it is a useful tool for comparing the absolute achievement of students.

If we consider GPA as a tool to measure the relative quality of students, it would only work within the same system (e.g., within the same school). This is fine for globally famous institutions, but high school GPAs are purportedly important even though there are very few high schools that a majority of college admissions officers would know by name. I earned a perfect, non-weighted 4.0 from Sir Francis Drake High School, but I think very few people would argue that holds the same weight as a 4.0 from Phillips Exeter Academy, even though Drake is a “California Distinguished School.” My 4.0 would also mean something different than a 4.0 from a rough inner-city high school (their’s would be a better indicator of resilience to adversity, for example). Context is important to communicating student performance and academic achievement, and GPA doesn’t provide any context.

In theory, GPA could be reasonable as a measure of a student’s absolute level of academic achievement, but it doesn’t appear that GPA fulfills this role in practice. If GPA should reflect absolute academic achievement, then we would expect that schools which perform poorly (e.g., on standardized tests, although those have their own slew of issues) would tend to give lower GPAs. In reality, you can find students with perfect or near perfect GPAs at any school, regardless of how well the school is preparing its students for employment or their next step in education. This can also be seen in “grade inflation,” which is the practice of better grades being given with each passing year. The data show that despite college grades improving significantly over the past few decades, students are more disengaged from learning, spend less time studying, and are less literate than in previous decades. I don’t think GPAs could be treated as an absolute measure unless some outside governing body was given control of assigning student grades, a prospect about as unlikely as scrapping the concept of the GPA altogether.

An alternative to GPAs: portfolios

While GPAs have their issues, we still need a way to assess students’ academic achievement, so what’s a better option? I think a reasonably implementable answer is student portfolios. A student portfolio is a collection of the student’s most exemplary work, selected and organized to emphasize the student’s strengths. Since the portfolio includes actual content they’ve generated, it allows the student to provide direct evidence of their mastery of subject matter. Ideally, project content included in a portfolio would be more closely aligned to work that would be performed in “the real world,” and would thus be much more useful than the result of an artificial test. The portfolio would therefore provide a much richer way for admissions officers or potential employers to evaluate the student than a transcript. Having students focus on building a portfolio rather than a GPA could also be pedagogically advantageous, because instead of “teaching to the test,” instructors would be incentivized to design assignments that would result in strong portfolio items, which would map more closely to genuine learning.  The single number provided by a GPA provides very little information, but part of a project included in a portfolio provides direct evidence of that student’s achievement.

Motivation for my personal website

What does this have to do with my making a personal website? So far in my academic career, I’ve benefited heavily from having a high GPA, but if I want to show potential future employers (among other people) that I have real content knowledge and experience, I need a portfolio of my own. As I add more content to the website, I hope it will play that role, and I’ll be able to share work I’ve done which has had outcomes more significant than a grade on a test. And at the very least, the website itself is inherently demonstrative of what I’ve learned about web development.

Assignments for week 1 of MIT Massive

Annotated Bibliography Proposal

I’m planning to take your advice to align my annotated bibliography with work I’m already doing. There are two directions I could take the assignment, and I would be open to feedback if you feel one would be more appropriate or more productive.

Option 1: A survey of competency based education (CBE)

I’m currently working on a small project with Sanjay Sarma to investigate competency based education or CBE, which is currently a trendy topic in K-16 education. The idea is to investigate how CBE is treated in areas where it has been around for a long time, especially where accurately assessing competency can be a matter of life or death (e.g., for surgeons or airline pilots), and to be able to draw lessons from those fields into K-16 education. When I originally spoke with Sanjay, I had intended to fold the work with him into this class, but I’m no longer 100% set on pursuing it here. This is largely because Sanjay is extremely busy (being named VP of Open Learning recently, in addition to his other responsibilities), and so I’m only able to meet with him every 4-6 weeks. Thus I’m not confident that I would have enough communication with him to be able to align work on the annotated bibliography with the project effectively. If I pursued this project, I would write my annotated bibliography on CBE, with a focus on fields in which CBE is not a new idea.

Option 2: Design-based  research (DBR) approach to improving MITxplore activities

I’m a mentor for MITxplore (www.mitxplore.org), which is a math outreach program for 4th – 6th graders which was started a few years ago by some of my engineering friends/colleagues (who have since graduated). When MITxplore first started, the group had grand visions for how research would be performed to ensure the efficacy of the activities we used, and with sufficient demonstration of the lessons’ effectiveness we would build a framework for the program that could expand beyond MIT. In reality, the “assessment team” stayed involved with MITxplore for only around a year, and we currently modify activities based on gut reactions and debriefing after the sessions. I think it could be very helpful to involve some elements of design-based research (DBR) in improving the activities we use, but I currently know very little about how DBR is implemented in practice. If I pursued this project, I would write my annotated bibliography on DBR, with a focus on applications similar to MITxplore.

Draft Participation Rubric

Personal Compass:

Broadly, I decided to take this class because I am interested in “learning science” (the study of learning) and I would like to be a better teacher. More specifically, I am interested in learning more about cMOOCs, which I know very little about, and in learning about the pedagogies and pitfalls of learning technologies designed to scale. I hope that if I have the opportunity to design curriculum for a MOOC or develop a learning technology intended to scale massively, that this course will provide me with the tools to be successful in that endeavor (or at least more successful than I would’ve been without the class).

Participation Commitments:

I plan to read news articles on education related topics and to share them (and my interpretation of them) with my classmates. I plan to stay active on Twitter over the course of class, and hopefully continue to use it after the class ends. In addition to sharing my thoughts, I’ve already found some new people to follow through whom I’ve found some interesting articles. I also plan to blog, on topics related to the course and topics not directly related to the course, and will share my writing with my classmates. I hope to participate in some online communities beyond the direct purview of the course (Twitter, blog) as well. I hope to read the work of my classmates, discuss with them, and form working relationships that last beyond the end of the course.

Participation Rubric

Criteria Exceeds Expectation Meets Expectation Underperforms Expectation
Read, share, and share thoughts on education articles Share 2+ articles per week, including blogging on my thoughts/reactions to them Share 1-2 articles per week, sometimes including thoughts/reactions Share ~1 article per week or less, only through Twitter with limited explanation of why I found the article interesting
Be an active Twitter user Share resources through Twitter, have dialogues through Twitter, participate in Twitter chats Use Twitter to find interesting articles and to share quick thoughts on resources I find Basically not using Twitter by the end of the course
Blog actively Write high quality blog posts on a diverse range of topics (akin to Matt Might at Utah) Write ~weekly blog posts on topics related to the course (e.g., response to an educational article) Write a small number of blog posts, none of which are particularly in depth
Explore Instructables or a similar online community Post my own DIY project to instructables and write about the experience Try to build one or two DIY projects and write about the experience Explore DIY projects on the site without actually undertaking any of them myself
Engage with classmates Leave the class having developed prospective collaborations with other students in the class Read classmates’ work, comment on it, and dialogue with them through the end of the course and beyond Occasionally read classmates’ work, occasionally leave short comments
Expand understanding of CBE (if I pursue that for grad assignment) Publish academic paper which provides a clear foundation for how to apply CBE in K-16 education Learn about CBE and write a report or long form article on what I’ve learned Learn about CBE, but create no significant content on the topic myself
Improve activity planning of MITxplore (if I pursue that for grad assignment) Implement DBR in MITxplore activity planning and demonstrate measurable improvement in program outcomes Make iteration of MITxplore activity planning more data driven Learn about DBR, but have no/limited implementation in MITxplore