An easy improvement to the MIT grad housing lotto

Disclaimer: MIT grad programs can be very insular (perhaps this is true of graduate programs in general), which causes an echo chamber effect leading grad students to be more acutely aware of problems that affect them than problems that affect people in general. The problem I’m discussing in this post is very much an MIT grad student “first world problem,” but even so, I’d argue it’s still worth trying to fix, and the following discussion might be applicable to other problems as well.

MIT has an atypically large graduate housing program: about a third of grad students live in MIT dorms, and the proportion among first year grad students is much higher. I lived in the dorms my first two years at MIT, while I only stayed in the dorms at UC Berkeley (my alma mater) for one year. That I lived in graduate dorms longer than I lived in undergrad dorms might surprise some, but it’s simply the result of how good of a deal the MIT grad dorms are. The dorms provide an extremely short commute, facilities personnel that are more responsive than any landlord I’ve dealt with, faster internet than I’ve had anywhere else, and you pay about market rate. Grad dorms are also set up more like apartments: your bathroom is only shared among a few people (not the entire hall) and you have a kitchen, so cooking for yourself is actually feasible. While all these features are quite nice, I feel the MIT graduate dorms have one major weak point: the lottery that is used to determine MIT grad housing assignments.

The process of how grad housing is assigned is described here, and while the process isn’t 100% transparent, enough information is provided that it’s possible to infer how the algorithm they use for room assignments works.

Here’s the experience from the perspective of the grad student looking for housing:

  1. You rank all the dorms (and room types) that you’d be willing to live in. You can also submit requested roommates along with your ranking. This ranking needs to be submitted by early May.
  2. A few weeks later you receive at most one room assignment. If you receive a room assignment and don’t take it, you pay a $250 penalty (not trivial on a grad student stipend). If you don’t receive a room assignment, you can add yourself to the wait list (at which point your fate is truly up in the air).
  3. If your requested roommates received the same room assignment that you did, you will (most likely) be assigned to the same room. If not, you are assigned random roommates.

There are two reasons that this system is frustrating to me. A minor frustration is that the delay between steps 1 and 2 is unnecessarily long (in terms of how complex the assignment algorithm is, although admittedly the delay is likely from bureaucratic reasons). My bigger complaint is with how roommate assignments are handled. When you’re looking for housing, one of the first (if not the first) things you determine is who you want to live with. If you have friends who you want to live with and try to live in the MIT dorms, you risk getting split up or having to pay a penalty, since you can’t indicate to the housing assignment process that you only want to live in the dorms if it’s with a specific roommate. This happened to me my second year at MIT: I got a room assignment while the friend I wanted to live with didn’t, but I stuck with MIT to avoid the penalty. (That experience might be the reason I’m writing this post, even though it happened years ago.)

The housing site implies that both of these issues are due to the “complexity of the algorithm.” They don’t explicitly tell us how the algorithm works, but they do provide three points about what the algorithm prioritizes and how to maximize your chances of getting a room assignment:

  • The first priority is to give room assignments to as many students as possible
  • The second priority is giving students higher room preferences
  • The more dorms/room types you rank, the higher your chances of getting an assignment

Here’s an algorithm (which I would characterize as simple, not complex) to place N students into housing assignments that is consistent with the three points above:

For each student, in order from 1 to N (student number determined randomly):

Perform a “placement search.” Go through the current student’s ranked rooms, until one with a vacancy is found. If a vacancy is found, give the current student that room assignment.

If no vacancies are found, perform a “push search.” Go through the current student’s ranked rooms, and for each room check the students already assigned there. If a student already assigned there has a vacancy in one of their lower ranked options, “push” that student to their lower choice and give the current student that room assignment.

If no vacancies are found and no students in the ranked rooms can be pushed to lower ranked options, the current student does not receive a room assignment.

Here is the algorithm expressed as pseudocode (I apologize that the formatting is terrible, I’ve wrestled with WordPress for long enough that I’ve given up on trying to make it readable for now):

For i=1:N
j = 0
placeSearch = true
pushSearch = false
While placeSearch = true
If student(i).rank(j) == empty
placeSearch = false
pushSearch = true
Else if checkVacancy(student(i).rank(j)) == true
Assign(student(i),student(i).rank(j))
placeSearch = false
Else j+=1
End
End
j = 0
While pushSearch = true
k = 0
If student(i).rank(j) == empty
pushSearch = false
Else
pushees = findAssigned(student(i).rank(j))
For pushees
pushSearch2 = true
While pushSearch2 = true
If pushee.rank(k) == empty
pushSearch2 = false
k = 0
Else if checkVacancy(pushee.rank(k)) == true
Assign(pushee,pushee.rank(k))
Assign(student(i),student(i).rank(j))
pushSearch2 = false
pushSearch = false
Else k+=1
End
End
End
j+=1
End
End
End

We can characterize the complexity of this algorithm using big O notation, and it has complexity O(n2) because it tries to place N students and for each of those students it might need to check order N students during the push search portion (n*n=n2). O(n2) is low complexity: the stable marriage algorithm used to match recently graduated MDs to residency programs is a famous example of an algorithm with complexity O(n2) that completes in a few minutes or less for tens of thousands of participants. Thus even the highly unoptimized version of the algorithm for MIT housing that I outlined above should complete in less than a minute. From a less mathematical standpoint, if the pseudocode for an algorithm can be written in a few dozen lines, it’s not that complex (in the colloquial sense of the word). A note: since roommate requests are submitted from the beginning, it is possible that they are factored into the algorithm that MIT uses as well, but given the language on the roommates section of the grad housing info website, this is unlikely.

I propose a simple change to the algorithm I’ve outlined above that would completely address the roommate portion of my complaint about the MIT grad housing lotto. If 2-4 students know they want to live together, they can apply for grad housing as a group. They would only be able to rank rooms that exactly accommodate the size of the group (e.g., a pair of students could apply for double rooms, but not triples or quads), and after that point the algorithm would treat them as a single student which takes up more than one spot. In the random number assignment, you could either give the group a single number, or you could give each individual student a number and assign the last (i.e., worst) number to the group overall (this would correspond to the idea that the group would only be assigned a room if each individual student would’ve been assigned a room had they applied separately). The complexity of the algorithm would remain the same after this change, and if an algorithm close to the one I proposed is being used already, minimal changes should be necessary in order to accommodate this group application option.

I think this change would be a big improvement to the MIT grad housing lotto, and I think it could also improve the experience of living in the dorms beyond just the application process. Before I listed many reasons why the grad dorms are superior to typical undergrad dorms, but there is one striking difference where undergrad dorms beat grad dorms: in the MIT grad dorms there is very little sense of community. While I attended some events hosted by the dorms while I lived there, I couldn’t tell you the names of any of my neighbors, and I didn’t form any lasting friendships with fellow dorm mates (besides those who I met through other channels, but happened to also live in the dorms). This is in stark contrast to my undergrad experience: a majority of the friends from Cal that I still keep in touch with, I met through living in the same dorm. I think my experience in the MIT grad dorms is typical, with the exception of students who get involved in dorm leadership.

There are many reasons this may be the case. Departments and research groups are more conducive to forming friendships as a grad student than as an undergrad, and as a grad student you spend a lot less time at home. However, I think the MIT grad dorms lack a feeling of community in part because many of the apartments are full of people who aren’t friends. If you aren’t friends with your roommates, you don’t want to spend as much time at home, and you don’t think of home as a place to hang out with friends (I experienced this my second year in the dorms). If the dorms were more accommodating of students trying to live with people they know they’ll get along with, I think it could provide a better environment for building a sense of community. With this change, MIT grad dorms might have more living rooms where people actually spend time together, and consequently might have more open doors and therefore more opportunities to meet neighbors.

While I think this change would be an improvement, I recognize that there are a number of reasons why this feature might be intentionally missing. It would create more demand for grad student housing, which I assume is already at capacity. Perhaps more importantly, by housing assignment’s zero sum nature, if grad housing begins serving the population that wants to room with their friends better, it will be serving the rest of the student population worse. It’s reasonable to me that the administration would be more concerned about grad housing serving the type of student who doesn’t have friends picked out to live with than those who do. That being said, if one of these reasons is true, it would be nice for MIT housing to be honest about their reasons for not implementing group applications rather than hiding behind the (rather condescending) explanation that “the algorithm is complex.”

MIT Massive Annotated Bibliography

I chose to complete my annotated bibliography on Competency Based Education (CBE), which is a trendy topic in education today. At the beginning of the semester, the extent of my knowledge of CBE was essentially that CBE approached teaching in a different manner than traditional “seat time” education. Traditionally, each student spends the same amount of time studying a topic, and at the end of that time they are assessed to see how well they learned it before moving on (they only receive more time to study a topic if they do egregiously poorly and are forced to retake the whole course). CBE aims to instead make level of mastery fixed, and students are given as much time as they need on a topic to reach that level of mastery. In this way, every student should be competent in all content areas before they graduate. This idea, especially in conjunction with ideas I’ve learned about differences in individuals speed of learning (e.g., see The End of Average by Todd Rose) seemed very compelling to me, and a promising direction for schools to move towards in the future. I was also interested in looking at CBE for my annotated bibliography because I am working on a small project with Prof. Sanjay Sarma to investigate how CBE is (or isn’t) used in areas where attaining competency is a matter of life or death (e.g., flying an airplane or performing a surgery). For this reason, the references I found are not all on traditional K-16 education, and many extend into other educational domains.


Chyung, Seung Youn, Donald Stepich, and David Cox. “Building a competency-based curriculum architecture to educate 21st-century business practitioners.” Journal of Education for Business 81.6 (2006): 307-314.

This article discusses applying CBE to curriculum design, in this particular case for business education. The paper provides a nice background to definitions of CBE and terms within CBE. They argue for definition of competencies within each field: “the generic dictionary scales are applicable to all jobs and none precisely.” The paper provides a framework/flow-chart for how one particular program developed a CBE-based curriculum for “Information and Performance Technology.”

 

Ennis, Michelle R. Competency models: a review of the literature and the role of the employment and training administration (ETA). Office of Policy Development and Research, Employment and Training Administration, US Department of Labor, 2008.

This review covers the use of competency models in assessing employees (or potential employees) and in determining what is necessary of an employee. Here competency is used to mean being able to apply knowledge, skills, abilities, behaviors and characteristics to performing a task. Furthermore, competencies are “specific personal qualities that are ‘causally related to effective and/or superior performance,’ are common across many settings and situations, and endure for some time.” A competency model is a tool for identifying the competencies needed for a specific role (i.e., a behavioral job description). Competency models are hierarchical: foundation which is common to many roles is common, and pieces more specific to a certain job are built on top of the foundational elements. Competency models provide a language and framework for determining the qualification of workers to move into different positions or how to improve their performance in the current position. Tying in assessments to competency based measures is important to successfully using competency based models. WGU pops up in the review as a higher education institution using competency models to try to train better teachers.

 

Frank, Jason R., et al. “Toward a definition of competency-based education in medicine: a systematic review of published definitions.” Medical teacher 32.8 (2010): 631-637.

This paper seeks to establish a clear, widely accepted definition of CBE within the medical field, which they approached by reviewing the medical literature on CBE (as well as searching Google). Four major themes within definitions of CBE are identified: organizing framework, rationale, contrast with time, and implementing CBE. Their proposed definition of CBE is as follows: “Competency-based education (CBE) is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learner-centredness.”

 

Hamilton, Neil W., and Sarah Schaefer. “What Legal Education Can Learn from Medical Education About Competency-Based Learning Outcomes Including Those Related to Professional Formation (Professionalism).” (2015).

This report discusses CBE’s adoption in the medical field in the context of how their lessons can help CBE in law (which adopted CBE later). One of the main takeaways are that core competencies of being a doctor were identified (professionalism, patient care and procedural skills, medical knowledge, etc.) and that these competencies are assessable. Another main takeaway was the idea of the “hidden curriculum”: that much of what medical students learn is not in lectures or didactics, but through interaction with attending physicians in rotations at hospitals and clinics. If the hidden curriculum doesn’t support the stated curriculum, it undermines the authority of the stated curriculum as representing the field’s true best interests.

 

Hodge, Steven. “The origins of competency-based training.” Australian journal of adult learning 47.2 (2007): 179.

This paper discusses the “societal” and “theoretical” origins of CBT. CBT arose in the US in the 50s, 60s, and 70s due to societal trends towards accountability and personalization. Apparently Sputnik was the impetus, and fear of Soviet technological superiority spurred the federal government to play a larger role in education and training. The key theoretical influences on CBT were behavioral psychology (due to competencies being observable behaviors) and systems theory (training as system). The paper also discusses specific theoretical contributions to CBT.

 

Jamieson, Lynn M. “Competency-Based Approaches to Sport Management.” Journal of Sport Management 1.1 (1987).

This paper discusses the require competencies of sport managers/professionals (not to be confused with athletes). Competencies for such a position are in areas such as business procedures, communications, facility/maintenance, governance, legality, management techniques, etc. This paper’s analysis of CBE in sports management uses a lot of Likert scoring and statistics, but is not particularly insightful with respect to CBE, I thought CBE applied to a relatively unique field would be more interesting.

 

Jones, Elizabeth A., and Richard A. Voorhees. “Defining and Assessing Learning: Exploring Competency-Based Initiatives. Report of the National Postsecondary Education Cooperative Working Group on Competency-Based Initiatives in Postsecondary Education. Brochure [and] Report.” (2002).

This is a long report (almost 200 pages) that outlines competency based education in the context of higher education. It is intended to be used as a resource for professors or administrators who want to start competency based initiatives at their institution. The report identifies four main categories of “strong practices” in CBE institutions: planning, selecting assessments, ensuring learning experiences are relevant to competencies, and reviewing assessment results for iteration. There are some good specific examples of what the authors identify as competencies (e.g., see exhibit 2, appendices). The report includes a comprehensive annotated bibliography with many further references that look valuable for deeper study. The majority of the report is used to describe 8 case studies. I should return to this to read them in further detail, but one observation I had was that (at least when this report was written) most of the assessments described were indistinguishable from traditional assessments (test, essays graded by rubric, etc.).

 

Lorenzo, George. “Western Governors University: How competency-based distance education has come of age.” Educational Pathways 6.7 (2007): 1-4.

This article reports on the success of Western Governors University (WGU) an institution which is entirely distance learning and competency based. Here competency based is stated as meaning that students earn their degrees by passing assessments rather than completing credit hours. WGU seems to target adult learners (the reported average student age is 38) and students who wouldn’t be well served by a traditional residential university education. The types of assessments used include objective tests (I’m not sure what this means), performance tests, portfolios and projects. As an example, for teaching licensure students must perform 12 weeks of live teaching under observation as one of their assessments. There is also an attempt to use industry standard certifications when they available (this is often the case in IT and health professions). This article paints a very promising picture of WGU (although I also get the vibe that it is written by WGU’s publicity department) and its use of CBE. One question I am left with is how effectively CBE can be implemented outside the program areas offered by WGU (business, IT, health and teaching). See also the WGU program guidebook, annotated below.

 

Malan, S. P. T. “The ‘new paradigm’ of outcomes-based education in perspective.” Journal of Family Ecology and Consumer Sciences/Tydskrif vir Gesinsekologie en Verbruikerswetenskappe 28.1 (2000).

This paper reviews the roots of OBE and attempts to put recent efforts in perspective. OBE is an approach to education where the focus is on successful demonstrations of learning sought: where the “what” and “whether” of learning is more important than the “when” and “how.” OBE dates back to the middle ages (craft guilds). CBE was big in the US in the 60s, and was based on six components: explicit learning outcomes, flexible time frame, varied teaching activities, criterion-referenced assessment, certification based demonstrations, and adaptable programs. The author argues that calling OBE a paradigm shift is overselling it, as there is insufficient research base to verify the claims of OBE, and that OBE is not fundamentally different paradigmatically from traditional educational approaches. OBE is discussed as a transformational (rather than transmissive) approach. OBE uses performance-based and authentic assessment strategies within the context of criterion-referenced assessment, which must integrate knowledge, skills and values.

 

McClarty, Katie Larsen, and Matthew N. Gaertner. “Measuring Mastery: Best Practices for Assessment in Competency-Based Education. AEI Series on Competency-Based Higher Education.” American Enterprise Institute for Public Policy Research (2015).

This report examines assessment in CBE, and sets a number of recommendations. The authors argue that it is critical to validate the assessment instrument used, that meaningful competency thresholds must be set which are based on multiple sources of evidence, and that assessment design should be drive by external validity (i.e., will employers care how students performed on these assessments?). These points are important because CBE cannot be successful without effective competency assessments. The article covers many specifics of the steps that should be taken in designing and validating an assessment.

 

Miller, Gregory E. “The assessment of clinical skills/competence/performance.” Academic medicine 65.9 (1990): S63-7.

This is a (quite well written) review of assessments used in medical education. The definitions used don’t map perfectly to many other CBE references in this bibliography, and the review distinguishes between knowledge, competency (meaning “applicable knowledge”), performance (meaning “demonstration in an artificial environment”), and action (what I would normally think of as performance, i.e., “doing the task in a real environment”). The review describes the different assessments developed and in use (at the time of writing) to infer all four of these characteristics of physicians in training. It is impressive how ahead of the curve the assessments described seem, given that this review is from 1990. The review also includes a number of insights gleaned from the field and the author’s experience: summative testing must be quite long to be accurate and ranking students is less important than determining if they’ve achieved a cut-off level of mastery. It is interesting to see that the author doesn’t know where professionalism should fit into medical education, an issue covered in depth in some of the other references in this bibliography (which had the benefit of being written almost two decades later).

 

Mueller, Paul S. “Incorporating professionalism into medical education: the Mayo Clinic experience.” The Keio journal of medicine 58.3 (2009): 133-143. APA

This paper discusses professionalism as a core physician competency. The paper claims that professionalism can be taught, learned and assessed (as opposed to being an intrinsic personal characteristic) and it is critical to do so as part of the education of certified medical professionals. As they argue that assessment of professionalism is critical, the authors also discuss methods for formative and summative assessment.

 

Scalese, Ross J., Vivian T. Obeso, and S. Barry Issenberg. “Simulation technology for skills training and competency assessment in medical education.” Journal of general internal medicine 23.1 (2008): 46-49.

This paper discusses the use of simulations in medical education and assessment, which has become more common partially due to a larger focus recently on competencies. They look at three types of simulations in particular: part task trainers, computer-enhanced mannequins, and virtual reality simulators. These technologies can help faculty save time and give students educational opportunities without exposing patients to novice practitioners. They also allow a proactive rather than ad hoc educational process (do whatever procedure needs doing at the time). In addition to benefits associated with teaching, simulations offer strong opportunities for assessment of competency.

 

Voorhees, Richard A. “Competency‐Based learning models: A necessary future.” New directions for institutional research 2001.110 (2001): 5-13.

This paper (chapter?) discusses the basic ideas of CBL as well as their potential to be implemented in higher education. Includes the popular pyramid diagram describing the relationships between competencies and other pieces in the conceptual learning model of CBE. The chapter discusses bundling and unbundling of competencies for different contexts. Competencies should be transparent, unambiguous and measurable. I should probably see if I can get a copy of the book from the library to do a more in depth reading.

 

Western Governors University. “Program Guidebook: Post-baccalaureate Teacher Preparation, Social Science (5-12).” (2015): 1-14.

This is a program guidebook for one of the teaching programs at WGU (see the article by Lorenzo above for details on WGU), which includes useful details and specifics about how programs are run at WGU. In addition to an explanation of what CBE means (in terms of relevance to the student), the guidebook also includes information on their mentoring approach, and how to connect with other students (since all coursework is completed online). The guidebook also covers the required assessments in detail, including supervised teaching demonstrations and a professional portfolio.


Through the process of writing my annotated bibliography, I learned a lot more about the definitions and frameworks associated with CBE, going beyond the basic idea that CBE is an alternative to seat time requirements. While not all definitions in the literature are 100% consistent, I think most of the authors I read would agree that a competency is the capability of applying knowledge, skills and abilities to the successful performance of a task. I also learned about competency models, which provide a framework for how different individual competencies can be bundled together, leading to successfully fulfilling a particular role or job (similar to how a single competency might bundle knowledge and skills in order to successfully perform a single task within that job). If one takes this approach to understanding what makes someone competent (or incompetent) at their job, it makes sense to design courses and educational opportunities so that they lead to their students acquiring valuable competencies.

Many of the references covered how to develop curriculum which uses CBE or how to implement CBE principles at an existing educational institution. Competencies associated with the course should be identified, and aligned with the learning objectives and course activities. Additionally, an assessment (or set of assessments) needs to be selected that accurately gauges the competency level of the student. Additionally, a level of achievement associated with successful acquisition of the competencies should be identified. Many references also covered more specific examples within a particular area of study, because while this general list of steps is simple, identifying competencies and developing assessments are very field specific and can be very involved.

I learned about some specific implementations of CBE in higher education, and while I found resources describing a number of different case studies, the case which is represented in my bibliography is Western Governors University (WGU). WGU seems to be successfully implementing CBE programs in business, IT, health and teaching, and mainly targets older students. CBE seems to match well with these fields (which have specific tasks that employees should be able to perform, somewhat in contrast to getting a degree in philosophy, for example), and there are many authentic performance assessments available that could demonstrate with confidence that a student has the necessary competencies without some seat time requirement. For older students, this model also seems very advantageous, as it avoids a full 4 years being required for a new degree when two or more of those years might be review of material they already learned through previous schooling or employment. WGU shows that CBE has promise in higher education, although there are definitely open questions for how the success at WGU would translate to more traditional institutions (e.g., residential colleges that primarily enroll young students).

It was very interesting to learn about the adoption of CBE in medical education, which many of the references I annotated were about. Medical education began to integrate aspects of CBE relatively early on, with the Miller review describing a well-established battery of CBE assessments already implemented and researched in 1990. This makes sense, because many of the tasks expected of a physician can be broken relatively cleanly into different competencies and the outcomes of their tasks are unambiguous and critically important (e.g., if a patient dies, that is a poor outcome potentially indicating a lack of competency). Somewhat unexpectedly however, one of the key features of CBE seems absent from medical education: there is still heavy reliance on seat time requirements. Medical school lasts at least four years for all M.D. students, and each residency program has a set number of years that must be completed before a physician can practice independently. It seems that some elements of CBE have been adopted in order to improve educational outcomes and alert institutions to individuals who need more training on a particular procedure, but the pieces which would accelerate the certification of doctors who don’t require this length of training are missing. For the critical competencies associated with medical doctors, it seems the powers that be still trust seat time requirements more than competency based performance assessments.

Besides the more factual information I learned, I came away with two main impressions of CBE. One of the impressions is that it seems very easy to come up with a curriculum in a CBE wrapper (i.e., identify desired competencies, choose activities aligned with those competencies, etc.) that is essentially identical to traditional education, just with an exit assessment rather than seat time requirements. While this might still sound appealing, in these cases (which I think are most commonly the application of CBE to K-12 education) it seems the assessments are often just essays that are graded with rubrics designed with competencies in mind. For most K-12 topics (with possible exceptions in areas that we’re already good at assessing in a very quantitative way, like math), I think CBE is still far from being able to claim the victories that have been achieved at small scale in higher education.

The other impression I got is that good assessments are key to the successful implementation of CBE, and developing good assessments can be very difficult. Related to the previous point about some CBE being traditional education in a different wrapper, if you have high confidence in the assessment being accurate, then the CBE class really starts to look more novel. However, I personally don’t think that essays graded with rubrics are a very effective way of assessing student competencies, although more research into the area showing their success could change my mind. The more exciting prospect to me is the development of assessments that begin to look more authentic and performance based, which I think represent a true departure from traditional educational structures. Of course, much further work would be needed to develop such assessments, and it’s not clear to me who could provide the resources for that development.

Overall I still think the basic ideas of CBE are compelling, and it makes sense to move towards that direction in K-16 as well as other areas of education. Even so, I think there is a lot of work that needs to happen before CBE will be more effective (and have more public trust) than traditional seat time education, especially in terms of assessment. I’m also wary of some attempts to implement CBE, which don’t seem to be that different from the systems they would replace, although they could still be helpful in incrementally developing curriculum, activities and assessments, so maybe there is little harm in those attempts. I’m excited to see what happens with CBE in the future, but I’m not convinced that if it becomes adopted in name, it will truly represent a shift in educational philosophy.

MIT Massive Network Participation

To frame my participation in the course, here are my participations commitments and personal compass from the beginning of MIT Massive:

Personal Compass:

Broadly, I decided to take this class because I am interested in “learning science” (the study of learning) and I would like to be a better teacher. More specifically, I am interested in learning more about cMOOCs, which I know very little about, and in learning about the pedagogies and pitfalls of learning technologies designed to scale. I hope that if I have the opportunity to design curriculum for a MOOC or develop a learning technology intended to scale massively, that this course will provide me with the tools to be successful in that endeavor (or at least more successful than I would’ve been without the class).

Participation Commitments:

I plan to read news articles on education related topics and to share them (and my interpretation of them) with my classmates. I plan to stay active on Twitter over the course of class, and hopefully continue to use it after the class ends. In addition to sharing my thoughts, I’ve already found some new people to follow through whom I’ve found some interesting articles. I also plan to blog, on topics related to the course and topics not directly related to the course, and will share my writing with my classmates. I hope to participate in some online communities beyond the direct purview of the course (Twitter, blog) as well. I hope to read the work of my classmates, discuss with them, and form working relationships that last beyond the end of the course.

 

Participation Rubric

I wrote the following rubric for myself, intending it to be written such that meeting/exceeding expectations in each category would lead to me participating in the course in a way that was well aligned with my own learning objectives. I’ve highlighted the category that I think best describes my participation during the course, and have elaborated on my thoughts on each criterion after the table.

Criteria Exceeds Expectation Meets Expectation Underperforms Expectation
Read, share, and share thoughts on education articles Share 2+ articles per week, including blogging on my thoughts/reactions to them Share ~1 article per week, sometimes including thoughts/reactions Share < 1 article per week, without discussing why I found it interesting
Be an active Twitter user Share resources through Twitter, have dialogues through Twitter, participate in Twitter chats Use Twitter to find interesting articles and to share quick thoughts on resources I find Limited Twitter use by the end of the course
Blog actively Write high quality blog posts on a diverse range of topics (akin to Matt Might at Utah) Write ~weekly blog posts on topics related to the course (e.g., response to an educational article) Write a small number of blog posts, none of which are particularly in depth
Explore Instructables or a similar online community Post my own DIY project to instructables and write about the experience Try to build one or two DIY projects and write about the experience Explore DIY projects on the site without actually undertaking any of them myself
Get involved in a cMOOC such as ds106 Start open ds106 in earnest, getting a good start so I can work through all the assignments over the summer Participate in some of the daily creates offered by ds106 Have no continued involvement with cMOOCs after my initial exploration
Engage with classmates Leave the class having developed prospective collaborations with other students in the class Read classmates’ work, comment on it, and dialogue with them through the end of the course and beyond Occasionally read classmates’ work, occasionally leave short comments
Expand understanding of CBE (annotated bibliography assignment) Publish academic paper related to CBE (although this would likely occur long after the course was completed) Learn about CBE and write a report or long form article on what I’ve learned (likely during the summer) Learn about CBE, but create no significant content on the topic myself

Read, share, and share thoughts on education articles

My reading habits (in terms of reading about education) changed very little when I started MIT Massive. During the course, I would read 2-3 articles per week that happened to show up on one of my RSS feeds. Perhaps I was reading a bit more broadly, as I found some articles from the MIT Massive Twitter network, but these readings didn’t lead to any blog posts or in-depth exchanges with classmates. For this reason, I would say that I underperformed my initial expectations on this criterion, although in retrospect I’m not sure how important this piece was. I still enjoy reading articles on education and expect to keep reading in the future, and maybe with more time it will become more natural for me to share my thoughts through Twitter or blogging, but I’m in no rush to get to that point.

Be an active Twitter user

I have stayed fairly active on Twitter throughout the course, using it to share my work for MIT Massive, the occasional article, and have had some exchanges with my classmates. I think one of the biggest obstacles to me using Twitter more actively was that it felt clunky to use it to try to write meaningful (and consequently long) comments (or maybe I just need more practice at using 140 characters). While I don’t think I’ll completely drop Twitter now that the course is done, I do expect that my Twitter usage will decrease significantly. I didn’t form enough connections outside the class to use it for too much other than #MITmassive related tweets, and I expect the hashtag activity to die down once the course is completed. Overall however, I’m happy with how much I participated through Twitter and would say that I met my expectations.

Blog actively

In many ways I think blogging was the most successful outcome of my enrollment in MIT Massive. I was proud to share some of my posts, which I felt accurately reflected my thinking and creativity, which feels like a small personal success of connectivist ideas for me. Perhaps more importantly, I really enjoyed the process of writing many of my blog posts, which I definitely can’t say is true of all the writing I have to do these days. I have a list of topics I’d still like to write up posts on, and plan to continue blogging into the foreseeable future in order to address those topics and more. Since all the long-form writing I did for the class ended up on the blog, I’ll go into more detail for some posts below, but overall I think my participation through blogging met and maybe exceeded my expectations. (It’s hard for me to settle exactly on “met” vs. “exceeded” because I’m happy with the quality of my posts, but I feel it’s too audacious for me to claim that I wrote great posts during the class; to a large extent that’s for others to decide.)

Explore Instructables or a similar online community

I wrote a blog post about Instructables which I’ll also comment on below, so for here all I’ll say is that I’m glad I set aside the time to work on Instructables, and the extent to which I did exceeded my expectations.

Get involved in a cMOOC such as ds106

I was quite taken with ds106 when I first learned about it (as I wrote about in a blog post that I’ll comment on further below) but since the first week it was covered as part of the MIT Massive curriculum I’ve only participated in one more Daily Create. I still visit the page occasionally and hope to participate again in the future, but overall I fell a bit short of my expectations for continued participation in ds106.

Engage with classmates

This was the most challenging piece for me in MIT Massive, and I had little interaction with my classmates aside from activities during the actual class hours and some exchanges on Twitter. I think part of the challenge is that engaging with classmates depends on my interests being well aligned with those of my classmates, which might not happen in a class with such a broad range of topics and a relatively small number of students. I really do wish I had engaged with my classmates more and formed some stronger connections that might’ve outlasted the course, and so from that perspective I definitely fell short of my expectations. That being said, writing about this now has brought up some ideas for me to mull over and I think I might be able to synthesize them into a blog post in the future.

Expand understanding of CBE (annotated bibliography assignment)

I chose to complete my annotated bibliography assignment on competency based education (CBE), which I have learned a good deal about since starting the course. Since my goals for understanding CBE have a time frame longer than the MIT Massive course, I cannot yet say where my accomplishments lie relative to expectations. I still hope and expect to produce more CBE-related work in addition to the annotated bibliography.

 

Below I’ve linked my five favorite posts I wrote over the course of MIT Massive, and included some brief comments specific to each:

Why I made a personal website

This post encapsulates some of my strongest feelings on changes that should happen in education, particularly related to assessment. While it isn’t directly related to the ideas of MIT Massive (at least, not the curriculum we were covering in class when I wrote the post) it got a decent amount of traction (because Justin shared the piece on Twitter) so I think many of my classmates read it, and the positive Twitter response helped encourage me to share more through blogging. When I wrote the post, all my personal website had was my CV, and since then I’ve only added a research section. I have more sections planned out which I intend to add, and eventually I’d like a very polished, complete(-looking) product.

Thoughts on connectivism and cMOOCs

This post was a reflection on my learning about connectivism and my participation is a couple ds106 activities, which is a connectivist MOOC on digital storytelling. I’m very glad that I was introduced to connectivism in a more in-depth (or maybe the phrase I’m looking for is “hands-on”) way than I had in the past, because after doing a few ds106 Daily Creates I was truly sold on connectivism. Perhaps ironically, I didn’t seem to find any classmates who were similarly enthused, and I’m not sure how many lasting connections I’ve made by taking this course.

In praise of quals

This piece also goes back to assessment, and I didn’t write it intending to have any connection to MIT Massive. I’ve included it here because in retrospect I think the reason I wrote the post is connected to ideas we learned about in MIT Massive. I wrote the piece because I would sometimes run “mock quals” during recitation for the course I TAed this semester (graduate heat transfer), and these sessions seemed to get lower attendance. I sent the post out to my students in an effort to convince more of them attend the mock quals, and I think having this specific audience in mind (as opposed to my other posts, which I wrote very generally, or in some cases more for myself) made it more effective at forming connections and engaging the students, which is something that many technologies for learning at scale are grappling with how to do effectively (that is, connecting to their ideal audience).

Making digital art: IM t-shirt design

In this post I wrote about the process of making a design for the new intramural champions T-shirt at MIT. I basically just like this post because it was associated with an opportunity for me to make some art, which I get to do much less often than I would like. I also feel like it’s a little funny (in how ineffective it is as an educational post), because from the level of detail I went into, all it could really hope to accomplish is to let readers know about one particular tool (the autotracer). In any case, the design did win the contest, so maybe you’ll see this design on shirts around MIT soon.

“First” impressions of Instructables

This post details my involvement in Instructables through two means: publishing instructions for how to make a honeycomb sock organizer and following instructions on how to turn a t-shirt into wall art. I had fun doing both of these things, and I’m glad that MIT Massive led to me finally participating in Instructables after having found out about it so long ago. I’ve gotten a few comments on the Instructable I published, so I’m hoping that people are actually out there making honeycombs and organizing socks!

 

To begin summarizing some of what I learned by taking MIT Massive, I think it’s appropriate to make an observation about my favorite blog posts. It’s no coincidence that four of the five blog posts I highlighted involved making something (that is, if you don’t count the act of writing a blog post itself as “making something”). Participating in this class has helped solidify my belief in constructionism, because my most meaningful participation corresponded to the times when I was making something. It makes intuitive sense that learning activities which closely match real-world performances (which often involve making something) will be more effective at teaching the pieces necessary for real-world performances. (It also makes intuitive sense that assessments which more closely match real-world performances will be more effective at identifying the skill and knowledge level of students.) I’m glad to have a small piece of evidence supporting this from my participation in this class, although I do wish that I had been aware of learning theories earlier, because it’s difficult to apply this lens to my many previous years of formal education retroactively. This will likely be the last official university class I take (I’m done with my PhD coursework requirements after this), but I’m sure it won’t be the last time I’m trying to learn something, and I’ll use this constructionist experience to inform decisions about how I try to learn in the future.

The “I used to think/but now I think” framework we used in the last class session was helpful for getting me to think about big takeaways from the class, and besides the constructionist piece mentioned above, I think I had two main takeaways. The first, which I shared during class, was that “I used to think that we were on the verge of a technologically driven revolution in education, but now I’m not so sure.” This one is interesting for me, because I also wrote about how technology has essentially no limits (which I still believe). While technology is limitless, because of the paradox of free, the paradox of routine and the curse of the familiar, the ways that technology tends to be used in education is very limited. So while I think there are technologies which already exist that could revolutionize education, the change realized is limited by human behavior rather than technology itself.

The second big takeaway I had from the course, which is perhaps more optimistic, is that “I used to think traditional institutions would be responsible for revolutionizing education, but now I’m not so sure.” While this statement might not seem optimistic, I think it is more hopeful when considering the previous piece. Education outside traditional institutions doesn’t need to be familiar, because it doesn’t need to be sold as traditional education. Revolutionary educational technology developed outside traditional institutions might be able to infiltrate (I hate the overuse of the word “disrupt” but perhaps it is appropriate here) them if their learning outcomes can be demonstrated in traditional (as well as more meaningful) assessments. I think we have the technological tools to revolutionize education, and we might be able to overcome the human behavior obstacle by (initially) avoiding the traditional channels for delivering education.

As a general takeaway, it seems like with all the focus and energy and money that learning at scale is getting, it’s inevitable that it will play an important role in the future of education. I think all three types (teacher- self- and peer-directed) have their strengths and will continue to see success, but I’m curious to see if they will lead to any real changes in education. Because as wide spread and acclaimed as they are, I feel that current implementations of learning at scale are mostly cases of “everything old is new again.”

 

“First” impressions of Instructables

Instructables is a DIY learning community that I’ve been interested in for a long time, and decided I would use MIT Massive as an excuse/motivation to actually try it out. The “First” in the title of this post is in quotes because I’ve been aware of Instructables for a long time (vaguely for many years, then more acutely when Eric Wilhelm gave the keynote at the MIT MechE Research Exhibition last year), but this is the first time I’ve actually made something from the site.

Trying an Instructable: T-shirt to wall art

I used to wear a lot of graphic t-shirts, and I would often find myself hesitant to throw out shirts that had gotten too ratty to wear, but which had designs that I really enjoyed. Many years ago I bookmarked this link, with the intention of following it in order to transform my unwearable shirts into wall art. Unfortunately, I never got around to actually buying a staple gun and stretcher bars, and in the mean time I have thrown out many t-shirts. This weekend I finally bought the tools I needed and turned one of my old shirts into wall art:

Instructables: T-shirt to wall art

Making wall art from the t-shirt was very easy, and in retrospect I’m sad that it took me so long to bite the bullet and buy the necessary tools. If I had done so earlier, I’d have a lot more art for my apartment (and I guess Goodwill would’ve had a few less t-shirts donated to them). In the future I’ll definitely be quicker to act on cool DIY projects that I find.

Publishing an Instructable: Honeycomb sock organizer

I also recently had an idea to help organize my socks: I wanted to make a honeycomb structure so each pair could have its own cell. I came up with a simple way to make it out of index cards, and I thought it would be a good opportunity to publish an Instructable of my own. You can read the full Instructable that I came up with here, and you can see the results below:

Instructable: Honeycomb sock organizer

Publishing my own Instructable was a lot more work. Documenting the process of making something is about as time consuming as actually making it. It’s also non-trivial to write the instructions so that they’re clear to follow (I’m not sure I did a good job of this on my instructable). I guess I’ll have to wait to see how the response is before I can really say if the extra effort was worth it, but even if no one else makes it, it is nice to have a record of my process. The only frustrating piece I would note is that I looked for DIY honeycomb organizers before making mine and didn’t find anything, but after I was done I found that you can buy honeycomb organizers for pretty cheap.

 

Making digital art: IM T-shirt design

For my entire college career, I’ve been heavily involved in intramural sports, both in my undergrad studies at Cal and in my graduate studies at MIT. At Cal, I was only involved as an athlete, but at MIT I’ve been more involved on the leadership side (MIT IMs are almost entirely organized by students). Currently we’re running a competition to design the new IM champion T-shirt, and I decided to submit an entry. While I’m no expert on design (just a sometimes artist), I thought it might be useful to document my process here in case other similarly non-expert designers would find it useful for their own work. Here’s the final product, if you’re interested in seeing how I got to this point, read on:

IM-shirt-LWBefore one can start with the design, you need to come up with the concept. For this design, I wanted to riff off the MIT mascot (the beaver, “nature’s engineer”) in a way that’s relevant to intramurals. I wanted a design that addressed that the T-shirt was the prize for winning a championship and that IMs are the most informal manifestation of organized sports in the college environment. The concept I came up with was a beaver holding a championship cup hewn from a log. After coming up with the concept itself, the first step in my design process was to sketch out a few different versions of it:

logo-sketches

Ideally the sketches in this part of the process would have more variation than the ones here. I guess it worked out that they were so similar in this case because I was happy enough with my concept that I didn’t need to iterate much. I did use the sketches to block out the design, and hone down exactly how I would try to draw the beaver (which is a lot harder to draw than a cup). It also helped me nail down a few details, like the cup handles being made from twigs and the neck of the cup having a beaver-chewed pattern. After the sketches, the next step was to draw a cleaner version in ink rather than pencil:

logo-drawing

If I was more handy with vector based image processing programs (programs like Adobe Illustrator or Inkscape), I think I could skip this step: I’d be able to make an iconic, high-contrast image entirely digitally. Since I don’t have a ton of experience with these programs, it’s much easier for me to draw an image the old school way. From the photo of the drawing, I do some post-processing (increasing the brightness and contrast so it looks as close to two tone as possible) and then feed it into autotracer.org a cool, free, online tool that “vectorizes” bitmap images. The output is something like this:

high_contrast_-_darkened

With the output from autotracer, I have a high contrast image more amenable to digital editing. For the T-shirt design, I wanted a white image and text on a black background, so some minimal editing gives the final T-shirt design:

IM-shirt-LW

I hope you like the design and enjoyed reading about the process, and if this inspires you to make some digital art of your own, I’d love to see it!

MIT Massive mid-term assignments

Updated Participation Rubric

Personal Compass:

Broadly, I decided to take this class because I am interested in “learning science” (the study of learning) and I would like to be a better teacher. More specifically, I am interested in learning more about cMOOCs, which I know very little about, and in learning about the pedagogies and pitfalls of learning technologies designed to scale. I hope that if I have the opportunity to design curriculum for a MOOC or develop a learning technology intended to scale massively, that this course will provide me with the tools to be successful in that endeavor (or at least more successful than I would’ve been without the class).

Participation Commitments:

I plan to read news articles on education related topics and to share them (and my interpretation of them) with my classmates. I plan to stay active on Twitter over the course of class, and hopefully continue to use it after the class ends. In addition to sharing my thoughts, I’ve already found some new people to follow through whom I’ve found some interesting articles. I also plan to blog, on topics related to the course and topics not directly related to the course, and will share my writing with my classmates. I hope to participate in some online communities beyond the direct purview of the course (Twitter, blog) as well. I hope to read the work of my classmates, discuss with them, and form working relationships that last beyond the end of the course.

Participation so far:

I feel that I have been fairly active in course participation so far. Perhaps less than I could be during the actual lecture hours, but I’ve taken good advantage of the online technologies we’re using in order to better “walk the walk” of learning at scale. I’ve had some back and forth with classmates on Twitter, although I haven’t interacted with many people outside the MIT Massive community. I think the most successful piece of my participation so far has been sharing thoughts through blog posts, of which I written half a dozen or so. Some posts are directly related to MIT Massive, while some are more tangentially related. I’ve enjoyed sharing my thoughts through the medium and it feels rewarding that I’ve gotten some comments, indicating that I’ve given people something to think about. I’m not sure how active I’ll stay after the course in all of the ways I’m participating now, but I do expect to keep blogging semi-regularly.

What I’ve learned:

If you believe that someone can truly learn something by reading about it (as opposed to experiencing it, or building something from it), then up to this point in the semester I’ve learned about the different types of learning at scale. I’ve learned that teacher-directed learning at scale (L@S) has some great examples of success, but that on aggregate it currently mostly serves students who already have access to the best educational resources. I’ve learned that while self-directed L@S is sometimes touted as a magical new panacea, it’s essential enabling technologies are decades old. I’ve learned about peer-directed L@S and I am most excited about its prospects, although I wonder how it can be better used to teach foundational curriculum (the traditional topics that are required in primary and secondary education). If you believe in learning by doing, then I have begun to learn how to connect with the community through writing tweets and blog posts.

What I hope to accomplish through the end of the course:

The main piece that I hope to accomplish through the end of the course is connecting more with my classmates. While I’ve had some conversations through Twitter and short discussions in class, nothing feels like it really “stuck” in the sense that I left the interaction feeling like I had a significantly deeper understanding of the topic at hand. I think the challenge is that writing is time consuming, and if both parties aren’t deeply interested in the topic, then the thread will be cut short due to lack of effort/input on one side. It’s easy for me to write long blog posts about things I care about, but unless someone else also cares deeply about the topic, I wouldn’t expect a long response from them. I’m not sure how to address this yet, but I’ll continue probing it in the following weeks.

Participation Rubric

Criteria Exceeds Expectation Meets Expectation Underperforms Expectation
Read, share, and share thoughts on education articles Share 2+ articles per week, including blogging on my thoughts/reactions to them Share ~1 article per week, sometimes including thoughts/reactions Share < 1 article per week, without discussing why I found it interesting
Be an active Twitter user Share resources through Twitter, have dialogues through Twitter, participate in Twitter chats Use Twitter to find interesting articles and to share quick thoughts on resources I find Limited Twitter use by the end of the course
Blog actively Write high quality blog posts on a diverse range of topics (akin to Matt Might at Utah) Write ~weekly blog posts on topics related to the course (e.g., response to an educational article) Write a small number of blog posts, none of which are particularly in depth
Explore Instructables or a similar online community Post my own DIY project to instructables and write about the experience Try to build one or two DIY projects and write about the experience Explore DIY projects on the site without actually undertaking any of them myself
Get involved in a cMOOC such as ds106 Start open ds106 in earnest, getting a good start so I can work through all the assignments over the summer Participate in some of the daily creates offered by ds106 Have no continued involvement with cMOOCs after my initial exploration
Engage with classmates Leave the class having developed prospective collaborations with other students in the class Read classmates’ work, comment on it, and dialogue with them through the end of the course and beyond Occasionally read classmates’ work, occasionally leave short comments
Expand understanding of CBE (annotated bibliography assignment) Publish academic paper related to CBE (although this would like occur long after the course was completed) Learn about CBE and write a report or long form article on what I’ve learned (likely during the summer) Learn about CBE, but create no significant content on the topic myself

 

 

Annotated Bibliography (work in progress)

This is a collection of references I’ve put together related to competency based education (CBE) along with some notes on what information is included in each reference. The annotations might not be all that useful to other people, they’re basically my personal notes. However, for anyone interested in CBE (who is as uninitiated as I am), you may find some helpful resources here.

 

Chyung, Seung Youn, Donald Stepich, and David Cox. “Building a competency-based curriculum architecture to educate 21st-century business practitioners.” Journal of Education for Business 81.6 (2006): 307-314.

This article discusses applying CBE to curriculum design, in this particular case for business education. The paper provides a nice background to definitions of CBE and terms within CBE. They argue for definition of competencies within each field: “the generic dictionary scales are applicable to all jobs and none precisely.” The paper provides a framework/flow-chart for how one particular program developed a CBE-based curriculum for “Information and Performance Technology.”

 

Frank, Jason R., et al. “Toward a definition of competency-based education in medicine: a systematic review of published definitions.” Medical teacher 32.8 (2010): 631-637.

This paper seeks to establish a clear, widely accepted definition of CBE within the medical field, which they approached by reviewing the medical literature on CBE (as well as searching Google). Four major themes within definitions of CBE are identified: organizing framework, rationale, contrast with time, and implementing CBE. Their proposed definition of CBE is as follows: “Competency-based education (CBE) is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learner-centredness.”

 

Hamilton, Neil W., and Sarah Schaefer. “What Legal Education Can Learn from Medical Education About Competency-Based Learning Outcomes Including Those Related to Professional Formation (Professionalism).” (2015).

This report discusses CBE’s adoption in the medical field in the context of how their lessons can help CBE in law (which adopted CBE later). One of the main takeaways are that core competencies of being a doctor were identified (professionalism, patient care and procedural skills, medical knowledge, etc.) and that these competencies are assessable. Another main takeaway was the idea of the “hidden curriculum”: that much of what medical students learn is not in lectures or didactics, but through interaction with attending physicians in rotations at hospitals and clinics. If the hidden curriculum doesn’t support the stated curriculum, it undermines the authority of the stated curriculum as representing the field’s true best interests.

 

Hodge, Steven. “The origins of competency-based training.” Australian journal of adult learning 47.2 (2007): 179.

This paper discusses the “societal” and “theoretical” origins of CBT. CBT arose in the US in the 50s, 60s, and 70s due to societal trends towards accountability and personalization. Apparently Sputnik was the impetus, and fear of Soviet technological superiority spurred the federal government to play a larger role in education and training. The key theoretical influences on CBT were behavioral psychology (due to competencies being observable behaviors) and systems theory (training as system). The paper also discusses specific theoretical contributions to CBT.

 

Jamieson, Lynn M. “Competency-Based Approaches to Sport Management.” Journal of Sport Management 1.1 (1987).

This paper discusses the require competencies of sport managers/professionals (not to be confused with athletes). Competencies for such a position are in areas such as business procedures, communications, facility/maintenance, governance, legality, management techniques, etc. This paper’s analysis of CBE in sports management uses a lot of Likert scoring and statistics, but is not particularly insightful with respect to CBE, I thought CBE applied to a relatively unique field would be more interesting.

 

Jones, Elizabeth A., and Richard A. Voorhees. “Defining and Assessing Learning: Exploring Competency-Based Initiatives. Report of the National Postsecondary Education Cooperative Working Group on Competency-Based Initiatives in Postsecondary Education. Brochure [and] Report.” (2002).

[to be annotated]

 

Malan, S. P. T. “The ‘new paradigm’ of outcomes-based education in perspective.” Journal of Family Ecology and Consumer Sciences/Tydskrif vir Gesinsekologie en Verbruikerswetenskappe 28.1 (2000).

This paper reviews the roots of OBE and attempts to put recent efforts in perspective. OBE is an approach to education where the focus is on successful demonstrations of learning sought: where the “what” and “whether” of learning is more important than the “when” and “how.” OBE dates back to the middle ages (craft guilds). CBE was big in the US in the 60s, and was based on six components: explicit learning outcomes, flexible time frame, varied teaching activities, criterion-referenced assessment, certification based demonstrations, and adaptable programs. The author argues that calling OBE a paradigm shift is overselling it, as there is insufficient research base to verify the claims of OBE, and that OBE is not fundamentally different paradigmatically from traditional educational approaches. OBE is discussed as a transformational (rather than transmissive) approach. OBE uses performance-based and authentic assessment strategies within the context of criterion-referenced assessment, which must integrate knowledge, skills and values.

 

Miller, Gregory E. “The assessment of clinical skills/competence/performance.” Academic medicine 65.9 (1990): S63-7.

[to be annotated]

 

Mueller, Paul S. “Incorporating professionalism into medical education: the Mayo Clinic experience.” The Keio journal of medicine 58.3 (2009): 133-143. APA

This paper discusses professionalism as a core physician competency. The paper claims that professionalism can be taught, learned and assessed (as opposed to being an intrinsic personal characteristic) and it is critical to do so as part of the education of certified medical professionals. As they argue that assessment of professionalism is critical, the authors also discuss methods for formative and summative assessment.

 

Scalese, Ross J., Vivian T. Obeso, and S. Barry Issenberg. “Simulation technology for skills training and competency assessment in medical education.” Journal of general internal medicine 23.1 (2008): 46-49.

This paper discusses the use of simulations in medical education and assessment, which has become more common partially due to a larger focus recently on competencies. They look at three types of simulations in particular: part task trainers, computer-enhanced mannequins, and virtual reality simulators. These technologies can help faculty save time and give students educational opportunities without exposing patients to novice practitioners. They also allow a proactive rather than ad hoc educational process (do whatever procedure needs doing at the time). In addition to benefits associated with teaching, simulations offer strong opportunities for assessment of competency.

 

Voorhees, Richard A. “Competency‐Based learning models: A necessary future.” New directions for institutional research 2001.110 (2001): 5-13.

This paper (chapter?) discusses the basic ideas of CBL as well as their potential to be implemented in higher education. Includes the popular pyramid diagram describing the relationships between competencies and other pieces in the conceptual learning model of CBE. The chapter discusses bundling and unbundling of competencies for different contexts. Competencies should be transparent, unambiguous and measurable. I should probably see if I can get a copy of the book from the library to do a more in depth reading.

In praise of quals

One of the requirements for earning a PhD from MIT (or any university, as far as I know) is to pass a qualifying exam, colloquially known as “quals.” Quals vary by department, and the mechanical engineering quals are particularly notorious. I think this is partially due to their difficulty and partially due to their unfamiliarity. They are difficult: despite most students preparing for months, the first time quals pass rate is somewhere around 75%. And they are different: a large part of quals is oral exams, where you solve a problem on a blackboard in front of a panel of professors who can ask you questions to probe your approach and your understanding.

To me, the style of quals, which is so different than how students are traditionally tested, should be a reason to celebrate quals rather than dread them. Almost every engineering class I’ve taken has had tests, and all of them used the same format: I sat in a room, silently and without access to online resources, and wrote down answers to questions on a piece of paper (even when the course was about the use of computer software). I find this testing tradition absurdly contrived: when in your working life will you be given a problem like the ones asked in written exams without access to colleagues or the internet? Admittedly written tests are good at assessing how well you can answer test questions, but I think it’s a far stretch to claim that performance on a written test will accurately assess one’s competence at solving problems on the job.

The oral exams associated with quals, however, are much more closely aligned to how problems are solved out in the world. You are asked a question that doesn’t necessarily have a single, clear-cut answer, and you have to convince others that you have a strong enough understanding of the underlying principles for them to trust that your answer is reasonable. This is what I do when I discuss issues regarding experiments with my lab mates or when I field questions after giving a talk at a conference. Which is not to say that the usefulness of this skill is limited to academia – this exercise is central to technical consulting, for example, and I imagine that it shows up in every non-rote engineering job out there. I think it’s only because oral exams are so resource intensive (~3 profs * ~30 minutes per student) that it’s so uncommon for us to be assessed this way.

I am not arguing that quals are without issue (their high stakes nature makes them inherently stressful), but I would encourage any students taking quals to be grateful for at least some aspects of the opportunity. The novel testing environment is closely aligned with your best interests as a PhD student: by the end of the process, you’ll have convinced a cadre of MIT professors, through your direct interaction with them, that you are deserving of a PhD from MIT. The same could not be said had you simply completed another battery of written exams.

 

Thoughts on connectivism and cMOOCs

I was first exposed to connectivism last year in Chris Dede’s class on educational technology. That class was when I first learned (at least in terms of formal definitions) about constructivism, behaviorism, and cognitivism, so I was (and am) by no means an expert on learning theories. That being said, the idea of connectivism was definitely striking to me. A person’s productivity strongly depends on the connections they have (e.g., when they encounter an obstacle, do they know someone who has an easy solution, or would they always try to work through the obstacle them self?) so to me it makes sense that learning and education should be tied to those connections as well. From a purely practical perspective, people don’t work in a vacuum in “the real world,” so it doesn’t make sense to artificially create a vacuum for them to work and be assessed in at school.

But while I learned about connectivism last semester, whatever I learned about cMOOCs (if anything) didn’t seem to stick. Like most people exposed to some education journalism, at the beginning of MIT Massive “MOOC” was synonymous to me with what I now know is referred to as an xMOOC. Perhaps it was just that I needed a concrete example, but after exploring the ds106 page (a cMOOC on digital storytelling) I am very taken with the idea. It paints a picture of how learning should be: driven by student interests, supported by a community, and focused on outputs that students can be proud of (basically all the positive features listed in the DML Connected Learning report).

At some point, perhaps when I have more time (haha, the mantra of a PhD student) or I’m more interested in having digital storytelling be a focus of my personal development, I’d like to go through the open ds106 course. As tangential participation through MIT Massive, I just decided to try a couple of the daily creates, short suggestions for making something that are put together by the ds106 community. I enjoyed both of the exercises I tried. They were relatively quick (~30 minutes each), allowed me to exercise creativity, and had a nice product at the end. I guess most importantly, it was fun to go through the process of making them.

For TDC1385 I wrote calligraphy of some Drake lyrics:

Drake Calligraphy

When I first saw the prompt, I thought the idea of writing out Drake lyrics in calligraphy was hillarious: I like the juxtaposition of the old and new. I was confused that none of the submitted responses were in calligraphy, until I read the prompt more closely. Even though it called for an illustration of the meaning, I found the literal interpretation of the prompt too funny to pass up.

For TDC1358 I designed a custom road sign:

RoadSign

This was inspired by a run in I had with a Boston driver earlier in the day (I guess technically it was a Boston passenger, he wasn’t driving the car) who cursed at my friend and I for not crossing the street when we had a “don’t walk” signal. Sometimes it feels like whatever you do, there’s a Boston driver who thinks you’re an idiot for it.

Is there a limit to technology’s role in the classroom of the future?

This week in MIT Massive, we had an activity where we were tasked with designing a school which takes advantage of personalized learning (where in this case, personalized learning means software like Knewton or Khan Academy). All the groups in our class assumed that such software can save time used to the teach basic content, freeing up time to participate in deeper learning experiences (e.g., project based learning). None of us designed a school where the personalized learning was both the means and the end, where the core student experience was solely interacting with their computer. Given that we seemed to agree that computers weren’t better at teaching everything, Justin asked if we see a limit to the role that technology can play in the classroom moving into the future. I have some thoughts on the topic that I’ll detail below, but to give you fair warning, my answer to this question is probably more of an excuse to wax philosophical than it is an attempt to reveal any insight about the role of technology in education.

The short version of my answer is that I don’t think there is any limit to the role technology can play in the classroom. In the future, I think it’s possible that a student could get a better education wearing a VR headset connected to a consumer computer than they could get in the best classrooms in the world today. And the version of “better education” that I’m claiming is all inclusive: they would have stronger content knowledge, better critical thinking skills, be more motivated to continue learning, have a stronger internal locus of control, and score higher on any other metric you can come up with. I believe this for primarily two reasons: 1. human consciousness does not transcend any physical laws and 2. technology continues to march forward at an unrelenting rate.

For me, the first point is almost self-evident. While consciousness is incredibly complicated, and the current framework for understanding it is beyond me, I do know that consciousness arises from something physical: the connection of neurons in our brain. While it’s true that there are many, many neurons in a human brain (on the order of 100 billion), there are also billions of transistors in the CPU of the laptop I’m writing this post on. From a physics perspective, I don’t see why we couldn’t connect a bunch of computers together and have consciousness arise: physically the infrastructure is there, we just haven’t made the right connections yet. A more visceral manifestation of this idea is that if you walked away from the movie Her thinking that [spoiler alert] Theodore’s love for Samantha (the AI) was at least as real as his love of Catherine (his ex-wife) had been, you were convinced that there is nothing magic about humans that gives us a monopoly on consciousness.

The second point is also fairly self-evident in current society. Moore’s law, the doubling of computational power every two or so years, is a well known phenomenon. A popular statement of the the effect of Moore’s law is that the computer in your pocket today is more powerful (by many orders of magnitude) than the computers used to land a man on the moon. While Moore’s law has been slowing down in the past few years, it doesn’t change the fact that computers are still becoming more powerful, and we can’t imagine a limit to what they’d be able to do (or perhaps more accurately, the limit is effectively infinity compared to what they can do today). One of my personal favorite examples of how far computers have come is AlphaGo, the computer program which was able to beat one of the world’s best Go players earlier this year, a task which wasn’t expected to happen until 2025 by most experts.

If these two points sound like they’re adding up to the singularity, the idea that when computer intelligence surpasses human intelligence it will lead to an infinitely accelerating wave of technological progress, I guess it’s because in essence that is what I’m arguing. If we accept these two points, eventually we’ll have computers that can pass the Turing test which are cheaper than a smartphone is today. Every student could have a personal AI tutor with the capacity for thought of a human and instant access to essentially all of human knowledge. They could interact with this tutor in a virtual reality with a level of resolution indistinguishable from the real world. They could also work on projects in teams where their teammates are AI teachable agents, operating to provide their student with an ideal learning experience (both in terms of learning content and non-cognitive skills). I would not be surprised if this vision is frightening to many, and if that’s the case for you I would remind you that in this post-singularity scenario, interacting with an AI is just as rich and complex as interacting with another real live human being.

Of course, even if this future is possible (or inevitable), it doesn’t do much to inform how technology should be used in education today. Any skill that a computer can teach more effectively than a human can also be performed more effectively by a computer than a human, meaning that skill is not useful for a human to have in order to be a productive member of society. In this future, there would be no need for humans to do work and we could live idle lives full of whatever leisurely pursuits we found most interesting. So while there’s no limit to the role technology could play in the post-singularity classroom of the future, that lack of a limit is irrelevant to the classroom of today.

PolyCraft World: between self and peer-directed learning?

I missed the first session of MIT Massive because I was in Phoenix, AZ at the MRS 2016 Spring Meeting, a large materials science conference. I spent most of the conference listening to talks about solar energy and new results for multi-junction PV cells, but I was also introduced to one educational technology that really struck me. At MRS I met Dr. Walter Voit, the creator of PolyCraft World, which is a mod (essentially an extension of functionality) for Minecraft. Minecraft, for the uninitiated, is a wildly popular video game that lets players explore and build in a stylized, blocky world, which itself has been implemented in classrooms as part of structured curricula. Even outside the classroom, I would characterize Minecraft as a learning community: it has a huge community generated wiki and countless hours of YouTube videos recording the in-game creations of Minecraft players.  

PolyCraft World adds additional materials and tools to the base Minecraft game, with a focus on chemistry and engineering. In Minecraft, players use materials like wood, stone, and iron, while in Polycraft World they also have the option to work with dozens of types of plastic, among other materials. Part of the educational value comes with the fact that working with the new materials is scientifically accurate: a chemical processing plant in PolyCraft World performs stoichiometrically balanced chemical reactions.

One of the more trailblazing plans Dr. Voit has for PolyCraft World is to teach some sections of intro organic chemistry at UT Dallas entirely through the game. Discussing this, the part I was most excited about was how much richer assessment could be in the software (e.g., build a reactor that can produce the desired chemical product, with a minimal footprint) than in a traditional pen and paper exam (e.g., identify the correct ratios of products and reactants in the following chemical equation). Reflecting back, now I’m also curious about how PolyCraft World bridges the space between the peer-directed learning of Minecraft and self-directed learning (if successful) of a game that can teach college-level organic chemistry. In Dan Meyer’s argument against personalized learning, he warns against the potential loss of synchronous learning, but I wonder if a system like this could provide some of the best of both worlds: scaffolding and motivation when the student wants to work on their own, but a global community of potential collaborators to undertake a grand project with when the student wants to work with others.