“Debunking” Salt Cases

Seeing something related to your field of study out in the “real world” can be a mixed experience. On one hand, it’s nice to have evidence that the topics you think about more than 99.9% of people (it would probably still be accurate with more 9s, but I’ll be conservative here) are important in contexts other than academia, and that non-academics do indeed sometimes think about them. On the other hand, it can be frustrating to see poorly reasoned/incorrect explanations about something related to your area of expertise.

I’ve encountered a few examples of frustrating heat transfer explanations out in the real world, and in this post I’ll look at one of them: Salt Cases. Salt Cases are cases designed for iPhones that are advertised as protecting the phone from both hot and cold. The basic premise alone is already suspicious, as being able to passively (i.e., without an external power source) maintain an intermediate temperature in both hot and cold environments is not easy to do (at least, not in steady state).

For cold protection, the explanation given is that the case is thermally insulating, and since the phone is dissipating a bit of power as heat (while it’s on, at least) it traps the heat in the phone and maintains it at a higher temperature than without the case. This explanation is completely reasonable – it’s the same reason why you get warmer when you put on more layers of clothes. You generate heat, and by adding the insulation of additional clothing layers, you trap that heat in your body, leading to a higher temperature.

The heat protection explanation isn’t very different: they say that the case is insulating (although with more of a focus on radiative heat transfer, which might be important if the case is in the sun, for example). At first blush, this might seem reasonable: the case is insulating, so if it’s hot outside you want to insulate the phone from that heat. However, when you consider this in combination with their cold protection explanation, things start to seem a little wonky. The phone is still dissipating power as heat when it’s hot out, and the same principle as before applies. So their explanation is akin to claiming that you’re going to wear a thick winter jacket in the middle of summer because you want it to insulate you from the hot weather outside.

Basically, it’s easy for something to be higher temperature than its ambient environment, you just need heat generation and insulation. It’s hard for something to be lower temperature than its ambient environment – with the exception of some exotic techniques, you need a refrigerant cycle and a decent amount of power.

That being said, they show a video where their case clearly leads to a lower temperature when two phones are left sitting in a car on a sunny day. There are a few explanations for why this could be:

  1. The cynical explanation is that they didn’t really leave the Salt Case phone in the car, they kept it somewhere else cooler and moved it into the car to take the video. As easy as this would be to do, I don’t think this is the correct explanation.
  2. They advertise their case as reflecting (and therefore not emitting) infrared light, and the thermometer they use is an infrared thermometer. When this type of thermometer is used to measure the temperature of something that doesn’t emit infrared, it’s doesn’t accurately measure the temperature of that object. I don’t think this is correct explanation either, because in the video they open the case, and it would be weird if the screen protector also reflected infrared.
  3. The case offers heat protection for a limited amount of time. Since phones don’t dissipate that much heat (if they did, you’d often notice that they felt hot in your hand), even if they were perfectly insulated the temperature would rise slowly. A back of the envelope calculation suggests a temperature rise on the order of 1 degree Fahrenheit per minute, so a well insulated phone might not start overheating in 30 minutes, but would after a few hours. I think this is the correct explanation.

Presumably the product’s claimed heat protection works to some degree, or Salt Cases would have many unhappy customers. But as best I can guess, they only provide passive heat protection over short periods of time. In any case, regardless of the effectiveness of their product, the explanation doesn’t really make sense – hopefully some day they’ll update their “technology” page to give an explanation more consistent with my understanding of heat transfer.

Levelized cost of wear

I consider this post a work-in-progress. If there’s any real merit to this concept, I’ll need to come back and clean up the post so that it’s coherent to someone who isn’t already familiar with all the ideas I use to build up the concept.

There’s an idea related to clothes shopping called “cost per wear.” It’s a simple idea for quantifying the cost of different items of clothing that’s (arguably) more useful than their sticker price. To find an item’s cost per wear, you simply divide its cost by the number of times you expect to wear it before getting rid of it:

CPW = \frac{cost}{times\ worn}

You can find plenty of articles about cost per wear (some good, and some which grossly misinterpret it), but in this post I’ll propose an extension to the idea. But before proposing the extension, I’ll recount how I ended up with it. I was first introduced to the idea of cost per wear by my undergraduate research advisor, who claimed that you should never spend less than $2000 on a suit (he always dressed very sharp). The reason he gave was cost per wear: if you buy a cheap suit, you’ll never want to wear it, and you’ll only get a few uses out of it before throwing it away. If you buy a nice suit, you’ll use it as much as you can (for dates, interviews, any vaguely formal event, etc.), and in the long run it will be cheaper – at least in terms of cost per wear. Putting numbers to this, there seems to be some wisdom in his claim. If I buy a cheapo suit for $120, it might last me two years, and I’ll only wear it when I absolutely need to wear a suit (maybe three times a year). This yields:

CPW_{cheapo\ suit} = \frac{\$120}{2 years* 3 wears/year}=\$20/wear

If I buy a primo suit for $2000, it could last me closer to ten years, and I’d want to wear it whenever I had the chance (maybe once a month). Therefore:

CPW_{primo\ suit} = \frac{\$2000}{10 years* 12 wears/year}=\$16.67/wear

Since the nice suit has a lower cost per wear, the argument goes, it’s actually the thriftier purchase. Somehow this idea came up in a discussion with a fellow grad student, and from our perspective it didn’t seem to paint an accurate picture of our situation if we were in the market for a new suit. A point that cost per wear misses is that a dollar in my pocket today is not the same as a dollar in my pocket ten years from now (something that grad students are acutely aware of, as in ten years we imagine our salaries will be at least triple what they are today).

There is another idea, called levelized cost of energy, or LCOE, which is used to calculate the effective cost of electricity from different sources – specifically as a way to compare renewable sources like wind and solar that are “free” once they are installed to conventional sources that required burning fuel which you have to pay for. The piece that’s relevant here is that even if you had a maintenance free solar panel that lasted forever, if you have to pay for the panel initially it’s LCOE would not be zero (i.e., the electricity it generates is not free). This is because even though you get infinite electricity from that panel, it’s spread out over infinite time. And when you consider inflation, the electricity that the panel generates in one or two hundred years has almost no value to you today.

I’m sure this explanation is insufficient for anyone not already familiar with LCOE, but in any case, that’s the idea I borrowed from to come up with “levelized cost of wear” or LCOW, an extension to cost per wear. LCOW is calculated as follows:

LCOW=\frac{cost}{\sum\limits_{t=1}^n \frac{wears/year}{(1+r)^t}}

where n is the number of years you expect the item to last and r is your “personal inflation rate” – nominally we can say that it’s how much you expect your salary to increase annually. LCOW takes into account how the value of money might change over time for an individual, whereas CPW does not. We can revisit the cheap vs. expensive suit using LCOW, using an r of 12% (this would be very high for most individuals, but for the PhD student example, it corresponds to about triple the salary in 10 years):

LCOW_{cheapo\ suit}=\frac{\$100}{\sum\limits_{t=1}^2 \frac{3}{(1.12)^t}}=\$23.67

LCOW_{primo\ suit}=\frac{\$2000}{\sum\limits_{t=1}^10 \frac{12}{(1.12)^t}}=\$29.49

When considering LCOW, the cheap suit is the thriftier purchase. Intuitively this makes sense – my friend and I will have plenty of time to go shopping for nice suits once we have real jobs. In the meantime, it’s a better use of our money to buy the cheapest suit that can get us through graduation.

Rocket League: the most authentic soccer video game ever made

A few weeks ago, at the recommendation of a friend, I started playing Rocket League. In Rocket League you control a rocket powered car in a game resembling indoor soccer: you score points by knocking a ball into the opposing team’s goal, and you try to prevent the other team from knocking the ball into yours. While I’m late to the party (Rocket League has been out for a bit over a year now), in these past few weeks it’s completely won me over: right now I’d say I’m spending the majority of my free time playing Rocket League.

I’ve also been playing some actual soccer this summer, and after playing soccer one day I realized why Rocket League is so compelling to me. Rocket League is the most authentic soccer video game I’ve ever played. A natural response might be: “That’s ridiculous! How could a game about cars playing soccer be more authentic than games like FIFA, which are about the actual sport of soccer?” I would not contest that watching a game played out in FIFA is more like watching an actual soccer game, but playing Rocket League captures the feeling of actually playing soccer better than any other video game I’ve played. I am not the first person to share this opinion, but nonetheless I thought I’d take this entry to explain my reasoning in a bit more depth.

The first reason is that the mechanics of Rocket League make the experience much closer to playing soccer. In an actual soccer game, when you want to shoot, pass, or clear the ball, all of these are accomplished by kicking the ball with your foot (well, usually contact is with the foot, but occasionally it could be with the leg, head, etc.). In FIFA-like video games, you perform the different kicks by pressing different buttons, e.g., [x] to pass or [y] to shoot, even though an actual soccer player kicks the ball in both cases, just in a different way. In Rocket League, the “kicks” are collisions, and all the contact between the car and the ball is physically simulated in the game. So whether you want to pass or shoot, in both cases you need to drive your car into the ball. Whether it’s a pass or a shot depends on how fast you strike the ball and at what angle, there’s no pass button you can press that will cause your car to knock the ball towards a teammate.

The result of these mechanics is that when you’re bearing down on a ball in front of the opponent’s goal in Rocket League, it feels like running up to shoot the ball in actual soccer. In both cases you’re acutely focused on how to strike the ball to maximize the chance of it going into the goal. In FIFA, on the other hand, you just press the “shoot” button and hope for the best (I’m probably underselling the control in FIFA a bit here, but you get the idea). Likewise, in Rocket League when you’re trying to clear a ball in front of your goal, you’re trying to get some part of the car between the goal and ball (similar to real soccer, but replace car with leg), not just mashing the “clear” button in the proximity of the ball.

The other big reason that Rocket League recreates an authentic soccer experience is the perspective. While not technically played in a 1st person view, Rocket League is 1st person in the sense that you only control one car, and your field of vision is limited to originating from around where that car is. This is different from FIFA-like games, which let you swap control of players so you’re typically in control of the player with the ball (on offense) or the player challenging the ball (on defense), and are viewed from a perspective similar to how professional soccer games are broadcast, so you can see all the relevant action around the ball.

In a real soccer game, what you do off the ball is very important: your position is critical to creating scoring opportunities for your team and shutting down chances for your opponent. This aspect of team play is largely out of your control in FIFA-like games: what the other players do on your team is automated. In Rocket League, since you only control your own car, you need to decide where to position yourself for the entire game, not just when you’re on the ball. And similar to real soccer, this positioning is very important: if you play like a 5-year-old plays soccer and just chase the ball the whole match, your team won’t have the structure to both attack and defend, and you’ll doom yourself to many losses.

The view that Rocket League is played in (psuedo 1st person) also makes the experience more authentic. In real soccer, situational awareness is an important skill to have, which is recreated in Rocket League by necessitating that you pan the camera up and down the field if you want to know where your teammates and opponents are. In FIFA-like games, the positions of everyone is simply laid out for you by the high viewing angle of the field. The Rocket League view also means you need to actually be able to judge distances and trajectories to know where you should head to intercept a ball or player (as in real soccer), whereas FIFA-like games put a glyph on the ground where the ball is going to land, since it would be intractable to assess a ball trajectory from the high viewing angle they use.

To me, this combination of mechanics and perspective end up making Rocket League so much more fun because it feels like actually playing soccer. In Rocket League it feels like you scored the goal, whereas in FIFA it feels like the character in the game scored, you just happened to tell him to shoot.

Windmills in Red Mars

Note: this post is based off a recitation problem I wrote when I was TAing graduate heat transfer last semester. At some point I hope to upload some of the problems I wrote, including this one, which should provide a more structural/mathematical approach to answering the question (rather than just the conceptual approach provided here).

In high school, I read the science fiction novel Red Mars by Kim Stanley Robinson, which chronicles the efforts of a colony of astronauts to terraform Mars (that is, make it more Earth-like, and therefore habitable to humans). Red Mars is fairly far towards the “hard” end of science fiction, meaning that the story is fueled, as much as possible, by actual science that we currently have a strong understanding of. To give recent Hollywood examples, The Martian is representative of hard science fiction, while Interstellar is representative of soft science fiction. This distinction is important because when something doesn’t click as being realistic (in terms of the science) in soft sci-fi, you’re expected to continue suspending disbelief, while if something doesn’t click as being realistic in hard sci-fi, it means the author messed up.

I read Red Mars many years ago, and in the intervening time I’ve forgotten most of the book. However, there is one plot point I still remember, because I disagreed with my high school science teacher about its scientific accuracy. At one point in the story, the colonists distribute windmills all over the surface of Mars which power heaters, with the intention of raising the average temperature of Mars. This post is concerned with the question of whether or not such a scheme could actually raise the surface temperature of Mars. Later in the story we find (spoiler alert) that the proponent of the windmill distribution plan didn’t care about their ability to raise the planet’s temperature, he wanted to use them to distribute plant life (algae? I don’t remember exactly) around Mars. That’s all fine and good, except that 1. some scientists in the story later find that the temperature of Mars has risen by some small (but apparently measurable) amount that they attribute to the windmills, and 2. the plan should be thermodynamically sound if we’re to believe that the other colonists could be convinced to go along with it, regardless of the plan’s true goals.

My science teacher argued that the plan is unsound, because wind is kinetic energy that will eventually be viscously dissipated and converted to heat anyways, so the windmills are pointless. My argument was that a Mars with windmills will be less windy on average (and therefore have less kinetic energy in its atmosphere) and because of conservation of energy its reasonable that the drop in kinetic energy would be accounted for by a rise in thermal energy (i.e., Mars would be at a higher temperature). For a long time I couldn’t reconcile these two viewpoints, however with some simplifying assumptions, we can capture the physics of the situation in a way that is compatible with both arguments.

Our Mars model has two forms of energy: kinetic energy (how windy it is on Mars) and thermal energy (how hot Mars is). The other important piece to know is that the only way Mars can exchange energy with the universe is by radiation: it absorbs energy radiated from the sun, and radiates out infrared radiation to the universe. The energy it receives from the sun should be constant, but the energy it radiates away is a function of its temperature (the hotter Mars gets, the more it radiates away). That means that Mars has some equilibrium temperature, at which the rate it loses energy to the universe is equal to the rate it absorbs energy from the sun. This system is self-balancing (or has negative feedback, to use control theory terminology): if Mars’ temperature rises above the equilibrium, it radiates away more than it absorbs, so the temperature lowers back to the equilibrium value. And vice-versa if Mars’ temperature drops below the equilibrium value. What is not defined by our simple model is how fast this happens, we just know that given enough time, Mars will reach its equilibrium temperature. From this perspective, my science teacher’s argument is correct, and even if you convert kinetic energy to thermal energy, in the long run the Martian temperature remains unchanged. However, this is only the long term behavior. When the windmills are deployed on Mars, they act to convert some of the kinetic energy in the wind to thermal energy. They will continue this conversion process until the wind on Mars reaches a new, lower, “post-windmill” value. Immediately after the windmills have converted the kinetic energy to thermal energy, they will have temporarily increased the average temperature of Mars.

So the full picture of this simple model for Mars is that with the windmills, the average temperature of Mars could be changed temporarily by converting kinetic energy to thermal energy, but in the long run the equilibrium temperature of Mars is determined by radiative exchange with the universe, which the windmills can’t (directly) address. In terms of how this fits into how realistic Red Mars is, I would say that on the first point, there could be a tiny (temporary) increase in temperature due to the windmills, but on the second point, at least one of the colonists should’ve had a strong enough understanding of thermodynamics to explain that this plan would ultimately be futile, and resources should be diverted to a different endeavor. So unfortunately, for this small corner of the story, it seems that Kim Stanley Robinson messed up.

An easy improvement to the MIT grad housing lotto

Disclaimer: MIT grad programs can be very insular (perhaps this is true of graduate programs in general), which causes an echo chamber effect leading grad students to be more acutely aware of problems that affect them than problems that affect people in general. The problem I’m discussing in this post is very much an MIT grad student “first world problem,” but even so, I’d argue it’s still worth trying to fix, and the following discussion might be applicable to other problems as well.

MIT has an atypically large graduate housing program: about a third of grad students live in MIT dorms, and the proportion among first year grad students is much higher. I lived in the dorms my first two years at MIT, while I only stayed in the dorms at UC Berkeley (my alma mater) for one year. That I lived in graduate dorms longer than I lived in undergrad dorms might surprise some, but it’s simply the result of how good of a deal the MIT grad dorms are. The dorms provide an extremely short commute, facilities personnel that are more responsive than any landlord I’ve dealt with, faster internet than I’ve had anywhere else, and you pay about market rate. Grad dorms are also set up more like apartments: your bathroom is only shared among a few people (not the entire hall) and you have a kitchen, so cooking for yourself is actually feasible. While all these features are quite nice, I feel the MIT graduate dorms have one major weak point: the lottery that is used to determine MIT grad housing assignments.

The process of how grad housing is assigned is described here, and while the process isn’t 100% transparent, enough information is provided that it’s possible to infer how the algorithm they use for room assignments works.

Here’s the experience from the perspective of the grad student looking for housing:

  1. You rank all the dorms (and room types) that you’d be willing to live in. You can also submit requested roommates along with your ranking. This ranking needs to be submitted by early May.
  2. A few weeks later you receive at most one room assignment. If you receive a room assignment and don’t take it, you pay a $250 penalty (not trivial on a grad student stipend). If you don’t receive a room assignment, you can add yourself to the wait list (at which point your fate is truly up in the air).
  3. If your requested roommates received the same room assignment that you did, you will (most likely) be assigned to the same room. If not, you are assigned random roommates.

There are two reasons that this system is frustrating to me. A minor frustration is that the delay between steps 1 and 2 is unnecessarily long (in terms of how complex the assignment algorithm is, although admittedly the delay is likely from bureaucratic reasons). My bigger complaint is with how roommate assignments are handled. When you’re looking for housing, one of the first (if not the first) things you determine is who you want to live with. If you have friends who you want to live with and try to live in the MIT dorms, you risk getting split up or having to pay a penalty, since you can’t indicate to the housing assignment process that you only want to live in the dorms if it’s with a specific roommate. This happened to me my second year at MIT: I got a room assignment while the friend I wanted to live with didn’t, but I stuck with MIT to avoid the penalty. (That experience might be the reason I’m writing this post, even though it happened years ago.)

The housing site implies that both of these issues are due to the “complexity of the algorithm.” They don’t explicitly tell us how the algorithm works, but they do provide three points about what the algorithm prioritizes and how to maximize your chances of getting a room assignment:

  • The first priority is to give room assignments to as many students as possible
  • The second priority is giving students higher room preferences
  • The more dorms/room types you rank, the higher your chances of getting an assignment

Here’s an algorithm (which I would characterize as simple, not complex) to place N students into housing assignments that is consistent with the three points above:

For each student, in order from 1 to N (student number determined randomly):

Perform a “placement search.” Go through the current student’s ranked rooms, until one with a vacancy is found. If a vacancy is found, give the current student that room assignment.

If no vacancies are found, perform a “push search.” Go through the current student’s ranked rooms, and for each room check the students already assigned there. If a student already assigned there has a vacancy in one of their lower ranked options, “push” that student to their lower choice and give the current student that room assignment.

If no vacancies are found and no students in the ranked rooms can be pushed to lower ranked options, the current student does not receive a room assignment.

Here is the algorithm expressed as pseudocode (I apologize that the formatting is terrible, I’ve wrestled with WordPress for long enough that I’ve given up on trying to make it readable for now):

For i=1:N
j = 0
placeSearch = true
pushSearch = false
While placeSearch = true
If student(i).rank(j) == empty
placeSearch = false
pushSearch = true
Else if checkVacancy(student(i).rank(j)) == true
Assign(student(i),student(i).rank(j))
placeSearch = false
Else j+=1
End
End
j = 0
While pushSearch = true
k = 0
If student(i).rank(j) == empty
pushSearch = false
Else
pushees = findAssigned(student(i).rank(j))
For pushees
pushSearch2 = true
While pushSearch2 = true
If pushee.rank(k) == empty
pushSearch2 = false
k = 0
Else if checkVacancy(pushee.rank(k)) == true
Assign(pushee,pushee.rank(k))
Assign(student(i),student(i).rank(j))
pushSearch2 = false
pushSearch = false
Else k+=1
End
End
End
j+=1
End
End
End

We can characterize the complexity of this algorithm using big O notation, and it has complexity O(n2) because it tries to place N students and for each of those students it might need to check order N students during the push search portion (n*n=n2). O(n2) is low complexity: the stable marriage algorithm used to match recently graduated MDs to residency programs is a famous example of an algorithm with complexity O(n2) that completes in a few minutes or less for tens of thousands of participants. Thus even the highly unoptimized version of the algorithm for MIT housing that I outlined above should complete in less than a minute. From a less mathematical standpoint, if the pseudocode for an algorithm can be written in a few dozen lines, it’s not that complex (in the colloquial sense of the word). A note: since roommate requests are submitted from the beginning, it is possible that they are factored into the algorithm that MIT uses as well, but given the language on the roommates section of the grad housing info website, this is unlikely.

I propose a simple change to the algorithm I’ve outlined above that would completely address the roommate portion of my complaint about the MIT grad housing lotto. If 2-4 students know they want to live together, they can apply for grad housing as a group. They would only be able to rank rooms that exactly accommodate the size of the group (e.g., a pair of students could apply for double rooms, but not triples or quads), and after that point the algorithm would treat them as a single student which takes up more than one spot. In the random number assignment, you could either give the group a single number, or you could give each individual student a number and assign the last (i.e., worst) number to the group overall (this would correspond to the idea that the group would only be assigned a room if each individual student would’ve been assigned a room had they applied separately). The complexity of the algorithm would remain the same after this change, and if an algorithm close to the one I proposed is being used already, minimal changes should be necessary in order to accommodate this group application option.

I think this change would be a big improvement to the MIT grad housing lotto, and I think it could also improve the experience of living in the dorms beyond just the application process. Before I listed many reasons why the grad dorms are superior to typical undergrad dorms, but there is one striking difference where undergrad dorms beat grad dorms: in the MIT grad dorms there is very little sense of community. While I attended some events hosted by the dorms while I lived there, I couldn’t tell you the names of any of my neighbors, and I didn’t form any lasting friendships with fellow dorm mates (besides those who I met through other channels, but happened to also live in the dorms). This is in stark contrast to my undergrad experience: a majority of the friends from Cal that I still keep in touch with, I met through living in the same dorm. I think my experience in the MIT grad dorms is typical, with the exception of students who get involved in dorm leadership.

There are many reasons this may be the case. Departments and research groups are more conducive to forming friendships as a grad student than as an undergrad, and as a grad student you spend a lot less time at home. However, I think the MIT grad dorms lack a feeling of community in part because many of the apartments are full of people who aren’t friends. If you aren’t friends with your roommates, you don’t want to spend as much time at home, and you don’t think of home as a place to hang out with friends (I experienced this my second year in the dorms). If the dorms were more accommodating of students trying to live with people they know they’ll get along with, I think it could provide a better environment for building a sense of community. With this change, MIT grad dorms might have more living rooms where people actually spend time together, and consequently might have more open doors and therefore more opportunities to meet neighbors.

While I think this change would be an improvement, I recognize that there are a number of reasons why this feature might be intentionally missing. It would create more demand for grad student housing, which I assume is already at capacity. Perhaps more importantly, by housing assignment’s zero sum nature, if grad housing begins serving the population that wants to room with their friends better, it will be serving the rest of the student population worse. It’s reasonable to me that the administration would be more concerned about grad housing serving the type of student who doesn’t have friends picked out to live with than those who do. That being said, if one of these reasons is true, it would be nice for MIT housing to be honest about their reasons for not implementing group applications rather than hiding behind the (rather condescending) explanation that “the algorithm is complex.”

MIT Massive Annotated Bibliography

I chose to complete my annotated bibliography on Competency Based Education (CBE), which is a trendy topic in education today. At the beginning of the semester, the extent of my knowledge of CBE was essentially that CBE approached teaching in a different manner than traditional “seat time” education. Traditionally, each student spends the same amount of time studying a topic, and at the end of that time they are assessed to see how well they learned it before moving on (they only receive more time to study a topic if they do egregiously poorly and are forced to retake the whole course). CBE aims to instead make level of mastery fixed, and students are given as much time as they need on a topic to reach that level of mastery. In this way, every student should be competent in all content areas before they graduate. This idea, especially in conjunction with ideas I’ve learned about differences in individuals speed of learning (e.g., see The End of Average by Todd Rose) seemed very compelling to me, and a promising direction for schools to move towards in the future. I was also interested in looking at CBE for my annotated bibliography because I am working on a small project with Prof. Sanjay Sarma to investigate how CBE is (or isn’t) used in areas where attaining competency is a matter of life or death (e.g., flying an airplane or performing a surgery). For this reason, the references I found are not all on traditional K-16 education, and many extend into other educational domains.


Chyung, Seung Youn, Donald Stepich, and David Cox. “Building a competency-based curriculum architecture to educate 21st-century business practitioners.” Journal of Education for Business 81.6 (2006): 307-314.

This article discusses applying CBE to curriculum design, in this particular case for business education. The paper provides a nice background to definitions of CBE and terms within CBE. They argue for definition of competencies within each field: “the generic dictionary scales are applicable to all jobs and none precisely.” The paper provides a framework/flow-chart for how one particular program developed a CBE-based curriculum for “Information and Performance Technology.”

 

Ennis, Michelle R. Competency models: a review of the literature and the role of the employment and training administration (ETA). Office of Policy Development and Research, Employment and Training Administration, US Department of Labor, 2008.

This review covers the use of competency models in assessing employees (or potential employees) and in determining what is necessary of an employee. Here competency is used to mean being able to apply knowledge, skills, abilities, behaviors and characteristics to performing a task. Furthermore, competencies are “specific personal qualities that are ‘causally related to effective and/or superior performance,’ are common across many settings and situations, and endure for some time.” A competency model is a tool for identifying the competencies needed for a specific role (i.e., a behavioral job description). Competency models are hierarchical: foundation which is common to many roles is common, and pieces more specific to a certain job are built on top of the foundational elements. Competency models provide a language and framework for determining the qualification of workers to move into different positions or how to improve their performance in the current position. Tying in assessments to competency based measures is important to successfully using competency based models. WGU pops up in the review as a higher education institution using competency models to try to train better teachers.

 

Frank, Jason R., et al. “Toward a definition of competency-based education in medicine: a systematic review of published definitions.” Medical teacher 32.8 (2010): 631-637.

This paper seeks to establish a clear, widely accepted definition of CBE within the medical field, which they approached by reviewing the medical literature on CBE (as well as searching Google). Four major themes within definitions of CBE are identified: organizing framework, rationale, contrast with time, and implementing CBE. Their proposed definition of CBE is as follows: “Competency-based education (CBE) is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learner-centredness.”

 

Hamilton, Neil W., and Sarah Schaefer. “What Legal Education Can Learn from Medical Education About Competency-Based Learning Outcomes Including Those Related to Professional Formation (Professionalism).” (2015).

This report discusses CBE’s adoption in the medical field in the context of how their lessons can help CBE in law (which adopted CBE later). One of the main takeaways are that core competencies of being a doctor were identified (professionalism, patient care and procedural skills, medical knowledge, etc.) and that these competencies are assessable. Another main takeaway was the idea of the “hidden curriculum”: that much of what medical students learn is not in lectures or didactics, but through interaction with attending physicians in rotations at hospitals and clinics. If the hidden curriculum doesn’t support the stated curriculum, it undermines the authority of the stated curriculum as representing the field’s true best interests.

 

Hodge, Steven. “The origins of competency-based training.” Australian journal of adult learning 47.2 (2007): 179.

This paper discusses the “societal” and “theoretical” origins of CBT. CBT arose in the US in the 50s, 60s, and 70s due to societal trends towards accountability and personalization. Apparently Sputnik was the impetus, and fear of Soviet technological superiority spurred the federal government to play a larger role in education and training. The key theoretical influences on CBT were behavioral psychology (due to competencies being observable behaviors) and systems theory (training as system). The paper also discusses specific theoretical contributions to CBT.

 

Jamieson, Lynn M. “Competency-Based Approaches to Sport Management.” Journal of Sport Management 1.1 (1987).

This paper discusses the require competencies of sport managers/professionals (not to be confused with athletes). Competencies for such a position are in areas such as business procedures, communications, facility/maintenance, governance, legality, management techniques, etc. This paper’s analysis of CBE in sports management uses a lot of Likert scoring and statistics, but is not particularly insightful with respect to CBE, I thought CBE applied to a relatively unique field would be more interesting.

 

Jones, Elizabeth A., and Richard A. Voorhees. “Defining and Assessing Learning: Exploring Competency-Based Initiatives. Report of the National Postsecondary Education Cooperative Working Group on Competency-Based Initiatives in Postsecondary Education. Brochure [and] Report.” (2002).

This is a long report (almost 200 pages) that outlines competency based education in the context of higher education. It is intended to be used as a resource for professors or administrators who want to start competency based initiatives at their institution. The report identifies four main categories of “strong practices” in CBE institutions: planning, selecting assessments, ensuring learning experiences are relevant to competencies, and reviewing assessment results for iteration. There are some good specific examples of what the authors identify as competencies (e.g., see exhibit 2, appendices). The report includes a comprehensive annotated bibliography with many further references that look valuable for deeper study. The majority of the report is used to describe 8 case studies. I should return to this to read them in further detail, but one observation I had was that (at least when this report was written) most of the assessments described were indistinguishable from traditional assessments (test, essays graded by rubric, etc.).

 

Lorenzo, George. “Western Governors University: How competency-based distance education has come of age.” Educational Pathways 6.7 (2007): 1-4.

This article reports on the success of Western Governors University (WGU) an institution which is entirely distance learning and competency based. Here competency based is stated as meaning that students earn their degrees by passing assessments rather than completing credit hours. WGU seems to target adult learners (the reported average student age is 38) and students who wouldn’t be well served by a traditional residential university education. The types of assessments used include objective tests (I’m not sure what this means), performance tests, portfolios and projects. As an example, for teaching licensure students must perform 12 weeks of live teaching under observation as one of their assessments. There is also an attempt to use industry standard certifications when they available (this is often the case in IT and health professions). This article paints a very promising picture of WGU (although I also get the vibe that it is written by WGU’s publicity department) and its use of CBE. One question I am left with is how effectively CBE can be implemented outside the program areas offered by WGU (business, IT, health and teaching). See also the WGU program guidebook, annotated below.

 

Malan, S. P. T. “The ‘new paradigm’ of outcomes-based education in perspective.” Journal of Family Ecology and Consumer Sciences/Tydskrif vir Gesinsekologie en Verbruikerswetenskappe 28.1 (2000).

This paper reviews the roots of OBE and attempts to put recent efforts in perspective. OBE is an approach to education where the focus is on successful demonstrations of learning sought: where the “what” and “whether” of learning is more important than the “when” and “how.” OBE dates back to the middle ages (craft guilds). CBE was big in the US in the 60s, and was based on six components: explicit learning outcomes, flexible time frame, varied teaching activities, criterion-referenced assessment, certification based demonstrations, and adaptable programs. The author argues that calling OBE a paradigm shift is overselling it, as there is insufficient research base to verify the claims of OBE, and that OBE is not fundamentally different paradigmatically from traditional educational approaches. OBE is discussed as a transformational (rather than transmissive) approach. OBE uses performance-based and authentic assessment strategies within the context of criterion-referenced assessment, which must integrate knowledge, skills and values.

 

McClarty, Katie Larsen, and Matthew N. Gaertner. “Measuring Mastery: Best Practices for Assessment in Competency-Based Education. AEI Series on Competency-Based Higher Education.” American Enterprise Institute for Public Policy Research (2015).

This report examines assessment in CBE, and sets a number of recommendations. The authors argue that it is critical to validate the assessment instrument used, that meaningful competency thresholds must be set which are based on multiple sources of evidence, and that assessment design should be drive by external validity (i.e., will employers care how students performed on these assessments?). These points are important because CBE cannot be successful without effective competency assessments. The article covers many specifics of the steps that should be taken in designing and validating an assessment.

 

Miller, Gregory E. “The assessment of clinical skills/competence/performance.” Academic medicine 65.9 (1990): S63-7.

This is a (quite well written) review of assessments used in medical education. The definitions used don’t map perfectly to many other CBE references in this bibliography, and the review distinguishes between knowledge, competency (meaning “applicable knowledge”), performance (meaning “demonstration in an artificial environment”), and action (what I would normally think of as performance, i.e., “doing the task in a real environment”). The review describes the different assessments developed and in use (at the time of writing) to infer all four of these characteristics of physicians in training. It is impressive how ahead of the curve the assessments described seem, given that this review is from 1990. The review also includes a number of insights gleaned from the field and the author’s experience: summative testing must be quite long to be accurate and ranking students is less important than determining if they’ve achieved a cut-off level of mastery. It is interesting to see that the author doesn’t know where professionalism should fit into medical education, an issue covered in depth in some of the other references in this bibliography (which had the benefit of being written almost two decades later).

 

Mueller, Paul S. “Incorporating professionalism into medical education: the Mayo Clinic experience.” The Keio journal of medicine 58.3 (2009): 133-143. APA

This paper discusses professionalism as a core physician competency. The paper claims that professionalism can be taught, learned and assessed (as opposed to being an intrinsic personal characteristic) and it is critical to do so as part of the education of certified medical professionals. As they argue that assessment of professionalism is critical, the authors also discuss methods for formative and summative assessment.

 

Scalese, Ross J., Vivian T. Obeso, and S. Barry Issenberg. “Simulation technology for skills training and competency assessment in medical education.” Journal of general internal medicine 23.1 (2008): 46-49.

This paper discusses the use of simulations in medical education and assessment, which has become more common partially due to a larger focus recently on competencies. They look at three types of simulations in particular: part task trainers, computer-enhanced mannequins, and virtual reality simulators. These technologies can help faculty save time and give students educational opportunities without exposing patients to novice practitioners. They also allow a proactive rather than ad hoc educational process (do whatever procedure needs doing at the time). In addition to benefits associated with teaching, simulations offer strong opportunities for assessment of competency.

 

Voorhees, Richard A. “Competency‐Based learning models: A necessary future.” New directions for institutional research 2001.110 (2001): 5-13.

This paper (chapter?) discusses the basic ideas of CBL as well as their potential to be implemented in higher education. Includes the popular pyramid diagram describing the relationships between competencies and other pieces in the conceptual learning model of CBE. The chapter discusses bundling and unbundling of competencies for different contexts. Competencies should be transparent, unambiguous and measurable. I should probably see if I can get a copy of the book from the library to do a more in depth reading.

 

Western Governors University. “Program Guidebook: Post-baccalaureate Teacher Preparation, Social Science (5-12).” (2015): 1-14.

This is a program guidebook for one of the teaching programs at WGU (see the article by Lorenzo above for details on WGU), which includes useful details and specifics about how programs are run at WGU. In addition to an explanation of what CBE means (in terms of relevance to the student), the guidebook also includes information on their mentoring approach, and how to connect with other students (since all coursework is completed online). The guidebook also covers the required assessments in detail, including supervised teaching demonstrations and a professional portfolio.


Through the process of writing my annotated bibliography, I learned a lot more about the definitions and frameworks associated with CBE, going beyond the basic idea that CBE is an alternative to seat time requirements. While not all definitions in the literature are 100% consistent, I think most of the authors I read would agree that a competency is the capability of applying knowledge, skills and abilities to the successful performance of a task. I also learned about competency models, which provide a framework for how different individual competencies can be bundled together, leading to successfully fulfilling a particular role or job (similar to how a single competency might bundle knowledge and skills in order to successfully perform a single task within that job). If one takes this approach to understanding what makes someone competent (or incompetent) at their job, it makes sense to design courses and educational opportunities so that they lead to their students acquiring valuable competencies.

Many of the references covered how to develop curriculum which uses CBE or how to implement CBE principles at an existing educational institution. Competencies associated with the course should be identified, and aligned with the learning objectives and course activities. Additionally, an assessment (or set of assessments) needs to be selected that accurately gauges the competency level of the student. Additionally, a level of achievement associated with successful acquisition of the competencies should be identified. Many references also covered more specific examples within a particular area of study, because while this general list of steps is simple, identifying competencies and developing assessments are very field specific and can be very involved.

I learned about some specific implementations of CBE in higher education, and while I found resources describing a number of different case studies, the case which is represented in my bibliography is Western Governors University (WGU). WGU seems to be successfully implementing CBE programs in business, IT, health and teaching, and mainly targets older students. CBE seems to match well with these fields (which have specific tasks that employees should be able to perform, somewhat in contrast to getting a degree in philosophy, for example), and there are many authentic performance assessments available that could demonstrate with confidence that a student has the necessary competencies without some seat time requirement. For older students, this model also seems very advantageous, as it avoids a full 4 years being required for a new degree when two or more of those years might be review of material they already learned through previous schooling or employment. WGU shows that CBE has promise in higher education, although there are definitely open questions for how the success at WGU would translate to more traditional institutions (e.g., residential colleges that primarily enroll young students).

It was very interesting to learn about the adoption of CBE in medical education, which many of the references I annotated were about. Medical education began to integrate aspects of CBE relatively early on, with the Miller review describing a well-established battery of CBE assessments already implemented and researched in 1990. This makes sense, because many of the tasks expected of a physician can be broken relatively cleanly into different competencies and the outcomes of their tasks are unambiguous and critically important (e.g., if a patient dies, that is a poor outcome potentially indicating a lack of competency). Somewhat unexpectedly however, one of the key features of CBE seems absent from medical education: there is still heavy reliance on seat time requirements. Medical school lasts at least four years for all M.D. students, and each residency program has a set number of years that must be completed before a physician can practice independently. It seems that some elements of CBE have been adopted in order to improve educational outcomes and alert institutions to individuals who need more training on a particular procedure, but the pieces which would accelerate the certification of doctors who don’t require this length of training are missing. For the critical competencies associated with medical doctors, it seems the powers that be still trust seat time requirements more than competency based performance assessments.

Besides the more factual information I learned, I came away with two main impressions of CBE. One of the impressions is that it seems very easy to come up with a curriculum in a CBE wrapper (i.e., identify desired competencies, choose activities aligned with those competencies, etc.) that is essentially identical to traditional education, just with an exit assessment rather than seat time requirements. While this might still sound appealing, in these cases (which I think are most commonly the application of CBE to K-12 education) it seems the assessments are often just essays that are graded with rubrics designed with competencies in mind. For most K-12 topics (with possible exceptions in areas that we’re already good at assessing in a very quantitative way, like math), I think CBE is still far from being able to claim the victories that have been achieved at small scale in higher education.

The other impression I got is that good assessments are key to the successful implementation of CBE, and developing good assessments can be very difficult. Related to the previous point about some CBE being traditional education in a different wrapper, if you have high confidence in the assessment being accurate, then the CBE class really starts to look more novel. However, I personally don’t think that essays graded with rubrics are a very effective way of assessing student competencies, although more research into the area showing their success could change my mind. The more exciting prospect to me is the development of assessments that begin to look more authentic and performance based, which I think represent a true departure from traditional educational structures. Of course, much further work would be needed to develop such assessments, and it’s not clear to me who could provide the resources for that development.

Overall I still think the basic ideas of CBE are compelling, and it makes sense to move towards that direction in K-16 as well as other areas of education. Even so, I think there is a lot of work that needs to happen before CBE will be more effective (and have more public trust) than traditional seat time education, especially in terms of assessment. I’m also wary of some attempts to implement CBE, which don’t seem to be that different from the systems they would replace, although they could still be helpful in incrementally developing curriculum, activities and assessments, so maybe there is little harm in those attempts. I’m excited to see what happens with CBE in the future, but I’m not convinced that if it becomes adopted in name, it will truly represent a shift in educational philosophy.

MIT Massive Network Participation

To frame my participation in the course, here are my participations commitments and personal compass from the beginning of MIT Massive:

Personal Compass:

Broadly, I decided to take this class because I am interested in “learning science” (the study of learning) and I would like to be a better teacher. More specifically, I am interested in learning more about cMOOCs, which I know very little about, and in learning about the pedagogies and pitfalls of learning technologies designed to scale. I hope that if I have the opportunity to design curriculum for a MOOC or develop a learning technology intended to scale massively, that this course will provide me with the tools to be successful in that endeavor (or at least more successful than I would’ve been without the class).

Participation Commitments:

I plan to read news articles on education related topics and to share them (and my interpretation of them) with my classmates. I plan to stay active on Twitter over the course of class, and hopefully continue to use it after the class ends. In addition to sharing my thoughts, I’ve already found some new people to follow through whom I’ve found some interesting articles. I also plan to blog, on topics related to the course and topics not directly related to the course, and will share my writing with my classmates. I hope to participate in some online communities beyond the direct purview of the course (Twitter, blog) as well. I hope to read the work of my classmates, discuss with them, and form working relationships that last beyond the end of the course.

 

Participation Rubric

I wrote the following rubric for myself, intending it to be written such that meeting/exceeding expectations in each category would lead to me participating in the course in a way that was well aligned with my own learning objectives. I’ve highlighted the category that I think best describes my participation during the course, and have elaborated on my thoughts on each criterion after the table.

Criteria Exceeds Expectation Meets Expectation Underperforms Expectation
Read, share, and share thoughts on education articles Share 2+ articles per week, including blogging on my thoughts/reactions to them Share ~1 article per week, sometimes including thoughts/reactions Share < 1 article per week, without discussing why I found it interesting
Be an active Twitter user Share resources through Twitter, have dialogues through Twitter, participate in Twitter chats Use Twitter to find interesting articles and to share quick thoughts on resources I find Limited Twitter use by the end of the course
Blog actively Write high quality blog posts on a diverse range of topics (akin to Matt Might at Utah) Write ~weekly blog posts on topics related to the course (e.g., response to an educational article) Write a small number of blog posts, none of which are particularly in depth
Explore Instructables or a similar online community Post my own DIY project to instructables and write about the experience Try to build one or two DIY projects and write about the experience Explore DIY projects on the site without actually undertaking any of them myself
Get involved in a cMOOC such as ds106 Start open ds106 in earnest, getting a good start so I can work through all the assignments over the summer Participate in some of the daily creates offered by ds106 Have no continued involvement with cMOOCs after my initial exploration
Engage with classmates Leave the class having developed prospective collaborations with other students in the class Read classmates’ work, comment on it, and dialogue with them through the end of the course and beyond Occasionally read classmates’ work, occasionally leave short comments
Expand understanding of CBE (annotated bibliography assignment) Publish academic paper related to CBE (although this would likely occur long after the course was completed) Learn about CBE and write a report or long form article on what I’ve learned (likely during the summer) Learn about CBE, but create no significant content on the topic myself

Read, share, and share thoughts on education articles

My reading habits (in terms of reading about education) changed very little when I started MIT Massive. During the course, I would read 2-3 articles per week that happened to show up on one of my RSS feeds. Perhaps I was reading a bit more broadly, as I found some articles from the MIT Massive Twitter network, but these readings didn’t lead to any blog posts or in-depth exchanges with classmates. For this reason, I would say that I underperformed my initial expectations on this criterion, although in retrospect I’m not sure how important this piece was. I still enjoy reading articles on education and expect to keep reading in the future, and maybe with more time it will become more natural for me to share my thoughts through Twitter or blogging, but I’m in no rush to get to that point.

Be an active Twitter user

I have stayed fairly active on Twitter throughout the course, using it to share my work for MIT Massive, the occasional article, and have had some exchanges with my classmates. I think one of the biggest obstacles to me using Twitter more actively was that it felt clunky to use it to try to write meaningful (and consequently long) comments (or maybe I just need more practice at using 140 characters). While I don’t think I’ll completely drop Twitter now that the course is done, I do expect that my Twitter usage will decrease significantly. I didn’t form enough connections outside the class to use it for too much other than #MITmassive related tweets, and I expect the hashtag activity to die down once the course is completed. Overall however, I’m happy with how much I participated through Twitter and would say that I met my expectations.

Blog actively

In many ways I think blogging was the most successful outcome of my enrollment in MIT Massive. I was proud to share some of my posts, which I felt accurately reflected my thinking and creativity, which feels like a small personal success of connectivist ideas for me. Perhaps more importantly, I really enjoyed the process of writing many of my blog posts, which I definitely can’t say is true of all the writing I have to do these days. I have a list of topics I’d still like to write up posts on, and plan to continue blogging into the foreseeable future in order to address those topics and more. Since all the long-form writing I did for the class ended up on the blog, I’ll go into more detail for some posts below, but overall I think my participation through blogging met and maybe exceeded my expectations. (It’s hard for me to settle exactly on “met” vs. “exceeded” because I’m happy with the quality of my posts, but I feel it’s too audacious for me to claim that I wrote great posts during the class; to a large extent that’s for others to decide.)

Explore Instructables or a similar online community

I wrote a blog post about Instructables which I’ll also comment on below, so for here all I’ll say is that I’m glad I set aside the time to work on Instructables, and the extent to which I did exceeded my expectations.

Get involved in a cMOOC such as ds106

I was quite taken with ds106 when I first learned about it (as I wrote about in a blog post that I’ll comment on further below) but since the first week it was covered as part of the MIT Massive curriculum I’ve only participated in one more Daily Create. I still visit the page occasionally and hope to participate again in the future, but overall I fell a bit short of my expectations for continued participation in ds106.

Engage with classmates

This was the most challenging piece for me in MIT Massive, and I had little interaction with my classmates aside from activities during the actual class hours and some exchanges on Twitter. I think part of the challenge is that engaging with classmates depends on my interests being well aligned with those of my classmates, which might not happen in a class with such a broad range of topics and a relatively small number of students. I really do wish I had engaged with my classmates more and formed some stronger connections that might’ve outlasted the course, and so from that perspective I definitely fell short of my expectations. That being said, writing about this now has brought up some ideas for me to mull over and I think I might be able to synthesize them into a blog post in the future.

Expand understanding of CBE (annotated bibliography assignment)

I chose to complete my annotated bibliography assignment on competency based education (CBE), which I have learned a good deal about since starting the course. Since my goals for understanding CBE have a time frame longer than the MIT Massive course, I cannot yet say where my accomplishments lie relative to expectations. I still hope and expect to produce more CBE-related work in addition to the annotated bibliography.

 

Below I’ve linked my five favorite posts I wrote over the course of MIT Massive, and included some brief comments specific to each:

Why I made a personal website

This post encapsulates some of my strongest feelings on changes that should happen in education, particularly related to assessment. While it isn’t directly related to the ideas of MIT Massive (at least, not the curriculum we were covering in class when I wrote the post) it got a decent amount of traction (because Justin shared the piece on Twitter) so I think many of my classmates read it, and the positive Twitter response helped encourage me to share more through blogging. When I wrote the post, all my personal website had was my CV, and since then I’ve only added a research section. I have more sections planned out which I intend to add, and eventually I’d like a very polished, complete(-looking) product.

Thoughts on connectivism and cMOOCs

This post was a reflection on my learning about connectivism and my participation is a couple ds106 activities, which is a connectivist MOOC on digital storytelling. I’m very glad that I was introduced to connectivism in a more in-depth (or maybe the phrase I’m looking for is “hands-on”) way than I had in the past, because after doing a few ds106 Daily Creates I was truly sold on connectivism. Perhaps ironically, I didn’t seem to find any classmates who were similarly enthused, and I’m not sure how many lasting connections I’ve made by taking this course.

In praise of quals

This piece also goes back to assessment, and I didn’t write it intending to have any connection to MIT Massive. I’ve included it here because in retrospect I think the reason I wrote the post is connected to ideas we learned about in MIT Massive. I wrote the piece because I would sometimes run “mock quals” during recitation for the course I TAed this semester (graduate heat transfer), and these sessions seemed to get lower attendance. I sent the post out to my students in an effort to convince more of them attend the mock quals, and I think having this specific audience in mind (as opposed to my other posts, which I wrote very generally, or in some cases more for myself) made it more effective at forming connections and engaging the students, which is something that many technologies for learning at scale are grappling with how to do effectively (that is, connecting to their ideal audience).

Making digital art: IM t-shirt design

In this post I wrote about the process of making a design for the new intramural champions T-shirt at MIT. I basically just like this post because it was associated with an opportunity for me to make some art, which I get to do much less often than I would like. I also feel like it’s a little funny (in how ineffective it is as an educational post), because from the level of detail I went into, all it could really hope to accomplish is to let readers know about one particular tool (the autotracer). In any case, the design did win the contest, so maybe you’ll see this design on shirts around MIT soon.

“First” impressions of Instructables

This post details my involvement in Instructables through two means: publishing instructions for how to make a honeycomb sock organizer and following instructions on how to turn a t-shirt into wall art. I had fun doing both of these things, and I’m glad that MIT Massive led to me finally participating in Instructables after having found out about it so long ago. I’ve gotten a few comments on the Instructable I published, so I’m hoping that people are actually out there making honeycombs and organizing socks!

 

To begin summarizing some of what I learned by taking MIT Massive, I think it’s appropriate to make an observation about my favorite blog posts. It’s no coincidence that four of the five blog posts I highlighted involved making something (that is, if you don’t count the act of writing a blog post itself as “making something”). Participating in this class has helped solidify my belief in constructionism, because my most meaningful participation corresponded to the times when I was making something. It makes intuitive sense that learning activities which closely match real-world performances (which often involve making something) will be more effective at teaching the pieces necessary for real-world performances. (It also makes intuitive sense that assessments which more closely match real-world performances will be more effective at identifying the skill and knowledge level of students.) I’m glad to have a small piece of evidence supporting this from my participation in this class, although I do wish that I had been aware of learning theories earlier, because it’s difficult to apply this lens to my many previous years of formal education retroactively. This will likely be the last official university class I take (I’m done with my PhD coursework requirements after this), but I’m sure it won’t be the last time I’m trying to learn something, and I’ll use this constructionist experience to inform decisions about how I try to learn in the future.

The “I used to think/but now I think” framework we used in the last class session was helpful for getting me to think about big takeaways from the class, and besides the constructionist piece mentioned above, I think I had two main takeaways. The first, which I shared during class, was that “I used to think that we were on the verge of a technologically driven revolution in education, but now I’m not so sure.” This one is interesting for me, because I also wrote about how technology has essentially no limits (which I still believe). While technology is limitless, because of the paradox of free, the paradox of routine and the curse of the familiar, the ways that technology tends to be used in education is very limited. So while I think there are technologies which already exist that could revolutionize education, the change realized is limited by human behavior rather than technology itself.

The second big takeaway I had from the course, which is perhaps more optimistic, is that “I used to think traditional institutions would be responsible for revolutionizing education, but now I’m not so sure.” While this statement might not seem optimistic, I think it is more hopeful when considering the previous piece. Education outside traditional institutions doesn’t need to be familiar, because it doesn’t need to be sold as traditional education. Revolutionary educational technology developed outside traditional institutions might be able to infiltrate (I hate the overuse of the word “disrupt” but perhaps it is appropriate here) them if their learning outcomes can be demonstrated in traditional (as well as more meaningful) assessments. I think we have the technological tools to revolutionize education, and we might be able to overcome the human behavior obstacle by (initially) avoiding the traditional channels for delivering education.

As a general takeaway, it seems like with all the focus and energy and money that learning at scale is getting, it’s inevitable that it will play an important role in the future of education. I think all three types (teacher- self- and peer-directed) have their strengths and will continue to see success, but I’m curious to see if they will lead to any real changes in education. Because as wide spread and acclaimed as they are, I feel that current implementations of learning at scale are mostly cases of “everything old is new again.”

 

“First” impressions of Instructables

Instructables is a DIY learning community that I’ve been interested in for a long time, and decided I would use MIT Massive as an excuse/motivation to actually try it out. The “First” in the title of this post is in quotes because I’ve been aware of Instructables for a long time (vaguely for many years, then more acutely when Eric Wilhelm gave the keynote at the MIT MechE Research Exhibition last year), but this is the first time I’ve actually made something from the site.

Trying an Instructable: T-shirt to wall art

I used to wear a lot of graphic t-shirts, and I would often find myself hesitant to throw out shirts that had gotten too ratty to wear, but which had designs that I really enjoyed. Many years ago I bookmarked this link, with the intention of following it in order to transform my unwearable shirts into wall art. Unfortunately, I never got around to actually buying a staple gun and stretcher bars, and in the mean time I have thrown out many t-shirts. This weekend I finally bought the tools I needed and turned one of my old shirts into wall art:

Instructables: T-shirt to wall art

Making wall art from the t-shirt was very easy, and in retrospect I’m sad that it took me so long to bite the bullet and buy the necessary tools. If I had done so earlier, I’d have a lot more art for my apartment (and I guess Goodwill would’ve had a few less t-shirts donated to them). In the future I’ll definitely be quicker to act on cool DIY projects that I find.

Publishing an Instructable: Honeycomb sock organizer

I also recently had an idea to help organize my socks: I wanted to make a honeycomb structure so each pair could have its own cell. I came up with a simple way to make it out of index cards, and I thought it would be a good opportunity to publish an Instructable of my own. You can read the full Instructable that I came up with here, and you can see the results below:

Instructable: Honeycomb sock organizer

Publishing my own Instructable was a lot more work. Documenting the process of making something is about as time consuming as actually making it. It’s also non-trivial to write the instructions so that they’re clear to follow (I’m not sure I did a good job of this on my instructable). I guess I’ll have to wait to see how the response is before I can really say if the extra effort was worth it, but even if no one else makes it, it is nice to have a record of my process. The only frustrating piece I would note is that I looked for DIY honeycomb organizers before making mine and didn’t find anything, but after I was done I found that you can buy honeycomb organizers for pretty cheap.

 

Making digital art: IM T-shirt design

For my entire college career, I’ve been heavily involved in intramural sports, both in my undergrad studies at Cal and in my graduate studies at MIT. At Cal, I was only involved as an athlete, but at MIT I’ve been more involved on the leadership side (MIT IMs are almost entirely organized by students). Currently we’re running a competition to design the new IM champion T-shirt, and I decided to submit an entry. While I’m no expert on design (just a sometimes artist), I thought it might be useful to document my process here in case other similarly non-expert designers would find it useful for their own work. Here’s the final product, if you’re interested in seeing how I got to this point, read on:

IM-shirt-LWBefore one can start with the design, you need to come up with the concept. For this design, I wanted to riff off the MIT mascot (the beaver, “nature’s engineer”) in a way that’s relevant to intramurals. I wanted a design that addressed that the T-shirt was the prize for winning a championship and that IMs are the most informal manifestation of organized sports in the college environment. The concept I came up with was a beaver holding a championship cup hewn from a log. After coming up with the concept itself, the first step in my design process was to sketch out a few different versions of it:

logo-sketches

Ideally the sketches in this part of the process would have more variation than the ones here. I guess it worked out that they were so similar in this case because I was happy enough with my concept that I didn’t need to iterate much. I did use the sketches to block out the design, and hone down exactly how I would try to draw the beaver (which is a lot harder to draw than a cup). It also helped me nail down a few details, like the cup handles being made from twigs and the neck of the cup having a beaver-chewed pattern. After the sketches, the next step was to draw a cleaner version in ink rather than pencil:

logo-drawing

If I was more handy with vector based image processing programs (programs like Adobe Illustrator or Inkscape), I think I could skip this step: I’d be able to make an iconic, high-contrast image entirely digitally. Since I don’t have a ton of experience with these programs, it’s much easier for me to draw an image the old school way. From the photo of the drawing, I do some post-processing (increasing the brightness and contrast so it looks as close to two tone as possible) and then feed it into autotracer.org a cool, free, online tool that “vectorizes” bitmap images. The output is something like this:

high_contrast_-_darkened

With the output from autotracer, I have a high contrast image more amenable to digital editing. For the T-shirt design, I wanted a white image and text on a black background, so some minimal editing gives the final T-shirt design:

IM-shirt-LW

I hope you like the design and enjoyed reading about the process, and if this inspires you to make some digital art of your own, I’d love to see it!

MIT Massive mid-term assignments

Updated Participation Rubric

Personal Compass:

Broadly, I decided to take this class because I am interested in “learning science” (the study of learning) and I would like to be a better teacher. More specifically, I am interested in learning more about cMOOCs, which I know very little about, and in learning about the pedagogies and pitfalls of learning technologies designed to scale. I hope that if I have the opportunity to design curriculum for a MOOC or develop a learning technology intended to scale massively, that this course will provide me with the tools to be successful in that endeavor (or at least more successful than I would’ve been without the class).

Participation Commitments:

I plan to read news articles on education related topics and to share them (and my interpretation of them) with my classmates. I plan to stay active on Twitter over the course of class, and hopefully continue to use it after the class ends. In addition to sharing my thoughts, I’ve already found some new people to follow through whom I’ve found some interesting articles. I also plan to blog, on topics related to the course and topics not directly related to the course, and will share my writing with my classmates. I hope to participate in some online communities beyond the direct purview of the course (Twitter, blog) as well. I hope to read the work of my classmates, discuss with them, and form working relationships that last beyond the end of the course.

Participation so far:

I feel that I have been fairly active in course participation so far. Perhaps less than I could be during the actual lecture hours, but I’ve taken good advantage of the online technologies we’re using in order to better “walk the walk” of learning at scale. I’ve had some back and forth with classmates on Twitter, although I haven’t interacted with many people outside the MIT Massive community. I think the most successful piece of my participation so far has been sharing thoughts through blog posts, of which I written half a dozen or so. Some posts are directly related to MIT Massive, while some are more tangentially related. I’ve enjoyed sharing my thoughts through the medium and it feels rewarding that I’ve gotten some comments, indicating that I’ve given people something to think about. I’m not sure how active I’ll stay after the course in all of the ways I’m participating now, but I do expect to keep blogging semi-regularly.

What I’ve learned:

If you believe that someone can truly learn something by reading about it (as opposed to experiencing it, or building something from it), then up to this point in the semester I’ve learned about the different types of learning at scale. I’ve learned that teacher-directed learning at scale (L@S) has some great examples of success, but that on aggregate it currently mostly serves students who already have access to the best educational resources. I’ve learned that while self-directed L@S is sometimes touted as a magical new panacea, it’s essential enabling technologies are decades old. I’ve learned about peer-directed L@S and I am most excited about its prospects, although I wonder how it can be better used to teach foundational curriculum (the traditional topics that are required in primary and secondary education). If you believe in learning by doing, then I have begun to learn how to connect with the community through writing tweets and blog posts.

What I hope to accomplish through the end of the course:

The main piece that I hope to accomplish through the end of the course is connecting more with my classmates. While I’ve had some conversations through Twitter and short discussions in class, nothing feels like it really “stuck” in the sense that I left the interaction feeling like I had a significantly deeper understanding of the topic at hand. I think the challenge is that writing is time consuming, and if both parties aren’t deeply interested in the topic, then the thread will be cut short due to lack of effort/input on one side. It’s easy for me to write long blog posts about things I care about, but unless someone else also cares deeply about the topic, I wouldn’t expect a long response from them. I’m not sure how to address this yet, but I’ll continue probing it in the following weeks.

Participation Rubric

Criteria Exceeds Expectation Meets Expectation Underperforms Expectation
Read, share, and share thoughts on education articles Share 2+ articles per week, including blogging on my thoughts/reactions to them Share ~1 article per week, sometimes including thoughts/reactions Share < 1 article per week, without discussing why I found it interesting
Be an active Twitter user Share resources through Twitter, have dialogues through Twitter, participate in Twitter chats Use Twitter to find interesting articles and to share quick thoughts on resources I find Limited Twitter use by the end of the course
Blog actively Write high quality blog posts on a diverse range of topics (akin to Matt Might at Utah) Write ~weekly blog posts on topics related to the course (e.g., response to an educational article) Write a small number of blog posts, none of which are particularly in depth
Explore Instructables or a similar online community Post my own DIY project to instructables and write about the experience Try to build one or two DIY projects and write about the experience Explore DIY projects on the site without actually undertaking any of them myself
Get involved in a cMOOC such as ds106 Start open ds106 in earnest, getting a good start so I can work through all the assignments over the summer Participate in some of the daily creates offered by ds106 Have no continued involvement with cMOOCs after my initial exploration
Engage with classmates Leave the class having developed prospective collaborations with other students in the class Read classmates’ work, comment on it, and dialogue with them through the end of the course and beyond Occasionally read classmates’ work, occasionally leave short comments
Expand understanding of CBE (annotated bibliography assignment) Publish academic paper related to CBE (although this would like occur long after the course was completed) Learn about CBE and write a report or long form article on what I’ve learned (likely during the summer) Learn about CBE, but create no significant content on the topic myself

 

 

Annotated Bibliography (work in progress)

This is a collection of references I’ve put together related to competency based education (CBE) along with some notes on what information is included in each reference. The annotations might not be all that useful to other people, they’re basically my personal notes. However, for anyone interested in CBE (who is as uninitiated as I am), you may find some helpful resources here.

 

Chyung, Seung Youn, Donald Stepich, and David Cox. “Building a competency-based curriculum architecture to educate 21st-century business practitioners.” Journal of Education for Business 81.6 (2006): 307-314.

This article discusses applying CBE to curriculum design, in this particular case for business education. The paper provides a nice background to definitions of CBE and terms within CBE. They argue for definition of competencies within each field: “the generic dictionary scales are applicable to all jobs and none precisely.” The paper provides a framework/flow-chart for how one particular program developed a CBE-based curriculum for “Information and Performance Technology.”

 

Frank, Jason R., et al. “Toward a definition of competency-based education in medicine: a systematic review of published definitions.” Medical teacher 32.8 (2010): 631-637.

This paper seeks to establish a clear, widely accepted definition of CBE within the medical field, which they approached by reviewing the medical literature on CBE (as well as searching Google). Four major themes within definitions of CBE are identified: organizing framework, rationale, contrast with time, and implementing CBE. Their proposed definition of CBE is as follows: “Competency-based education (CBE) is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learner-centredness.”

 

Hamilton, Neil W., and Sarah Schaefer. “What Legal Education Can Learn from Medical Education About Competency-Based Learning Outcomes Including Those Related to Professional Formation (Professionalism).” (2015).

This report discusses CBE’s adoption in the medical field in the context of how their lessons can help CBE in law (which adopted CBE later). One of the main takeaways are that core competencies of being a doctor were identified (professionalism, patient care and procedural skills, medical knowledge, etc.) and that these competencies are assessable. Another main takeaway was the idea of the “hidden curriculum”: that much of what medical students learn is not in lectures or didactics, but through interaction with attending physicians in rotations at hospitals and clinics. If the hidden curriculum doesn’t support the stated curriculum, it undermines the authority of the stated curriculum as representing the field’s true best interests.

 

Hodge, Steven. “The origins of competency-based training.” Australian journal of adult learning 47.2 (2007): 179.

This paper discusses the “societal” and “theoretical” origins of CBT. CBT arose in the US in the 50s, 60s, and 70s due to societal trends towards accountability and personalization. Apparently Sputnik was the impetus, and fear of Soviet technological superiority spurred the federal government to play a larger role in education and training. The key theoretical influences on CBT were behavioral psychology (due to competencies being observable behaviors) and systems theory (training as system). The paper also discusses specific theoretical contributions to CBT.

 

Jamieson, Lynn M. “Competency-Based Approaches to Sport Management.” Journal of Sport Management 1.1 (1987).

This paper discusses the require competencies of sport managers/professionals (not to be confused with athletes). Competencies for such a position are in areas such as business procedures, communications, facility/maintenance, governance, legality, management techniques, etc. This paper’s analysis of CBE in sports management uses a lot of Likert scoring and statistics, but is not particularly insightful with respect to CBE, I thought CBE applied to a relatively unique field would be more interesting.

 

Jones, Elizabeth A., and Richard A. Voorhees. “Defining and Assessing Learning: Exploring Competency-Based Initiatives. Report of the National Postsecondary Education Cooperative Working Group on Competency-Based Initiatives in Postsecondary Education. Brochure [and] Report.” (2002).

[to be annotated]

 

Malan, S. P. T. “The ‘new paradigm’ of outcomes-based education in perspective.” Journal of Family Ecology and Consumer Sciences/Tydskrif vir Gesinsekologie en Verbruikerswetenskappe 28.1 (2000).

This paper reviews the roots of OBE and attempts to put recent efforts in perspective. OBE is an approach to education where the focus is on successful demonstrations of learning sought: where the “what” and “whether” of learning is more important than the “when” and “how.” OBE dates back to the middle ages (craft guilds). CBE was big in the US in the 60s, and was based on six components: explicit learning outcomes, flexible time frame, varied teaching activities, criterion-referenced assessment, certification based demonstrations, and adaptable programs. The author argues that calling OBE a paradigm shift is overselling it, as there is insufficient research base to verify the claims of OBE, and that OBE is not fundamentally different paradigmatically from traditional educational approaches. OBE is discussed as a transformational (rather than transmissive) approach. OBE uses performance-based and authentic assessment strategies within the context of criterion-referenced assessment, which must integrate knowledge, skills and values.

 

Miller, Gregory E. “The assessment of clinical skills/competence/performance.” Academic medicine 65.9 (1990): S63-7.

[to be annotated]

 

Mueller, Paul S. “Incorporating professionalism into medical education: the Mayo Clinic experience.” The Keio journal of medicine 58.3 (2009): 133-143. APA

This paper discusses professionalism as a core physician competency. The paper claims that professionalism can be taught, learned and assessed (as opposed to being an intrinsic personal characteristic) and it is critical to do so as part of the education of certified medical professionals. As they argue that assessment of professionalism is critical, the authors also discuss methods for formative and summative assessment.

 

Scalese, Ross J., Vivian T. Obeso, and S. Barry Issenberg. “Simulation technology for skills training and competency assessment in medical education.” Journal of general internal medicine 23.1 (2008): 46-49.

This paper discusses the use of simulations in medical education and assessment, which has become more common partially due to a larger focus recently on competencies. They look at three types of simulations in particular: part task trainers, computer-enhanced mannequins, and virtual reality simulators. These technologies can help faculty save time and give students educational opportunities without exposing patients to novice practitioners. They also allow a proactive rather than ad hoc educational process (do whatever procedure needs doing at the time). In addition to benefits associated with teaching, simulations offer strong opportunities for assessment of competency.

 

Voorhees, Richard A. “Competency‐Based learning models: A necessary future.” New directions for institutional research 2001.110 (2001): 5-13.

This paper (chapter?) discusses the basic ideas of CBL as well as their potential to be implemented in higher education. Includes the popular pyramid diagram describing the relationships between competencies and other pieces in the conceptual learning model of CBE. The chapter discusses bundling and unbundling of competencies for different contexts. Competencies should be transparent, unambiguous and measurable. I should probably see if I can get a copy of the book from the library to do a more in depth reading.