Assigning & Assessing

The chair of the faculty has announced Emergence Academic Regulations that include “Alternate Grades” (PE, NE, and IE) for all undergraduate and graduate full-term subjects this semester. (Note: a more detailed version of policy is available here.) Given this policy, what follows are a set of resources and ideas associated with facilitating assessments and assignments in the coming weeks of the semester.

Reconsidering (Summative) Assessments

Revise assessment and assignments in response to revised learning outcomes and remote learning context. As you revise your learning outcomes, you should also consider what assessments, assignments, and activities contribute to achieving these outcomes in a remote-learning scenario. Some activities that work well in a traditional classroom or lab may not work well remotely. Consider why you had originally wanted students to participate in these activities and identify alternative ways to achieve those intended learning outcomes in a remote context.

For assignments or projects initially designed as collaborative, consider possible barriers (e.g., students distributed across time zones),  whether the same goals can be met individually, or if students can effectively collaborate through a combination of tools (e.g., Zoom, Slack, Google Docs, Dropbox).

 

General Considerations

For any class size

For all classes

For design-based classes

For small to medium classes

For medium to large classes

Tools

Case Studies at MIT

3.016 | Mathematics for Materials Science and Engineers

2.086 | Numerical Computation for Mechanical Engineers

2.007 | Design and Manufacturing I

 

General Considerations

Learning outcomes

Identify the learning outcomes that you have deemed  as most important for your course and prioritize those in your summative assessment strategy. 

Pass/No Record Grading

The function of summative assessment is to measure the degree to which students have met the learning outcomes. Because all classes this semester are graded pass/no record, assessments no longer need to be able to differentiate student degrees of achievement on a 5-point grade scale (A/B/C/D/F). As you review and revise your assessments, consider omitting aspects that are primarily used to discriminate between, for example, A- and B- student work, and focus on adapting aspects that determine whether students have sufficiently mastered the material to pass the course.

Types of Assessments

Prioritize frequent, low-stakes assessments/feedback. Higher stakes assessments (e.g., exams) on a specified day will present challenges for some students — depending on their circumstances on the day in question — and may be less necessary in a semester in which we are doing P/NR grading.

If you cancel your final exam and replace it with other assessments, you must notify Mary Callahan by the end of this week so that the Registrar can remove your exam from the final exam schedule.  In addition, you will need to notify your students of your change in plans by the end of this week (4/3).

Student Motivation

Consider ways to keep students engaged. Connect course concepts to real-world examples or topics your students are interested in, use puzzles and friendly competitions (with prizes if possible, see the “3.016 Case Study” below) to encourage students to engage deeply with course content, showcase unique problem solving approaches used by students, and provide many opportunities for student-student and student-teaching staff interaction.

Student Input

Partner with students in designing an appropriate final assessment. This creates an opportunity for students to communicate the technical capabilities available to them which might impact their work on the final assessment and builds student buy-in to changes to the class.

Student Well-being

Consider how different options for summative assessment might affect student well-being. This table nicely summarizes how different summative assessment strategies impact student well-being (University of California at UC Davis).

Code of Conduct

Consider asking your students to adhere to a code of conduct for your course/assessments. Studies have shown that students that are asked to sign a code of conduct prior to taking an exam, are less likely to engage in academically dishonest behavior1.  For an MIT example, see the “2.086 Case Study” below.  Alternatively, you can include a cover page in your exam that specifies the collaboration policy, instructions for exam completion and submission, and the following statement:

“Please sign the following statement to verify that you have read and abide by the code of conduct for this subject:  By affixing my name to this paper, I affirm that I have executed the examination in accordance with the code of conduct for this subject: ______________.”

For any class size

For all classes

Series of quizzes instead of a final

  • Offers multiple data points for assessment.
  • Lowers the stress of having  a single final summative assessment.
  • You can always reserve one question in each quiz for previously tested content  to make sure students have not forgotten it.

Omit remaining quizzes and exams

  • Reweight and revise your problem sets to make them more substantial.
  • Side-steps the challenge of retrofitting an old exam format to remote teaching or creating a new assessment from scratch.
  • Example at MIT: 18.03 (Differential Equations) has decided to omit the remainder of the quizzes and the final altogether. Final grades will be based on the problem sets and the first midterm (which was given when students were on campus).

Peer teaching

If your class size allows, have students present a concept or a solution to a problem to their peers. You might structure this by assigning students specific concepts or problems, or you might ask students to develop their presentation based on certain criteria. This activity could also be done during structured recitations. Doing this activity synchronously has the added benefit that students can be encouraged to offer presenters feedback and ask clarifying questions.
If your class is large, this activity above could be done asynchronously by having students create videos using their cell phones or laptop cameras  to post and share with the class.

For design-based classes

  • If collaborative, check-in with project teams to see if there are logistical barriers (e.g., students in multiple, distant time zones) that will impede student progress. Consider re-shuffling teams or allowing students to work on the project individually.
  • If the design project is paper and/or computationally based, major modifications may not be needed.
  • If the original intent of the design project was to have students create physical prototypes, consider alternatives that may still allow students to develop some of the intended skills. For example, can students create reasonable prototypes out of low-cost everyday materials? If not, consider whether or not drawing and sketching (by hand or using software) the prototype or creating an analytical prototype can still help students meet some of the goals that you intended for them at the beginning of the semester (see the “2.007 Case Study” below).
  • Often physical prototypes are developed so that they can be evaluated against certain benchmarks or criteria. If criteria were not already specified for students, consider having students develop metrics for a “successful” design instead. 
  • Consider whether or not a series of case studies of different products on the market could substitute for some of the learning goals of the design project.
  • Set up a schedule to check-in with students/student teams on a regular basis to answer questions and hear about their progress. TAs can assist with this.

For small to medium classes

Self-evaluations with or without student portfolios 

Students complete a  self-evaluation in which they reflect on their work/progress towards course learning outcomes. This reflection exercise can be extremely beneficial for students because it encourages them to: 1) focus on learning rather than grades; 2) encourages metacognition - a lifelong learning skill; 3) allows you to individualize feedback. 

  • The self-evaluation can be accompanied by a selection of student work (quizzes, homeworks, etc) or portfolio (see this example of how to use student portfolios and self-reflections from Olin College).
  • After students submit their reflections, a 1:1 meeting with you to discuss their progress and next steps is a nice complement even if this is done in the place of a final exam. 
  • If you choose this option, recognize that students may not be accustomed to such an exercise and will benefit from examples and structure. 
  • Some things that students can be asked to reflect on include: assignments completed; in-class activities and discussions; how they approach their work; how they interact with their peers; knowledge, skills and abilities that they feel that they have acquired in the course; “lightbulb” moments when they have connected course content to other aspects of their lives (e.g., other courses, day-to-day scenarios).

1:1 oral exam

Some MIT faculty already use oral exams to assess student learning (see examples here and here). This mode of assessment would translate well to a remote teaching and learning situation as oral exams could take place on any video chat platform (e.g., Zoom, Webex, etc).  You’ll want to carefully consider your departmental and subject culture before implementing an oral exam as an entirely new format may create anxiety for students.

  • Develop a set of well-structured questions and an interview protocol to assess a student’s level of understanding and facilitate a great dialogue. One approach is to map out the learning objectives that the exam is targetting: 1) arrange them from conceptually simple to conceptually complex; 2) indicate which objectives are prerequisite for other objectives; and 3) write questions that map onto these objectives. Now you have a map to follow during your oral exam. 
  • Start with a simpler question, a question that you expect most students would be able to productively respond to, to put students at ease. If the student gets the question correct, ask a harder question. If the student doesn’t get the more difficult question correct, ask a simpler but related question. Use your map to adjust the questions to the student’s level. Treat the oral exam more as an intellectual discussion rather than a classical exam.
  • Reserve a couple of days on your calendar for the oral exams and allocate ~20 min per student. Use an online scheduling tool like Doodle to create time slots where students can sign up. 

Student presentation/demonstration:

In place of an in-class exam, identify presentations or demonstrations students could do to demonstrate that they have met their intended learning outcomes. 

  • Presentations or demonstrations have the added benefit of providing a more authentic assessment that encourages students to engage with messy, real-world scenarios.
  • Students can do multiple takes if recording asynchronously.
  • Creates an opportunity for student-student interaction with peer review and feedback on presentations (see “Peer Teaching” above).
  • Examples:
    • Present on a topic of their choice related to course content or selected from an instructor-curated list.
    • Digital poster session with peer feedback.
    • Musical composition, digital artwork, recording of a performance.

Project or essay

As with presentations/demonstrations, students can complete a project or essay in place of an exam to demonstrate achieving learning outcomes. Examples might include:

  • Writing a document common in your discipline (e.g., policy memo, artistic statement, grant application, excerpt from an academic article).
  • Annotated bibliography
  • Scientific or academic abstract
  • Analysis of a case study, data set, or data visualization
  • Draft a fact sheet or Wikipedia entry for a general audience on a key topic of the course

For medium to large classes

The following strategy can also be used in small to medium classes for exams during the semester (i.e., it cannot be used during Finals Period). 

Take home or open-book exams 

  • Can lower the stress of final exams, especially during what already is a stressful time.
  • Allows for higher-order-thinking questions that prompt students to analyze, evaluate and synthesize.
  • Allows for open-ended questions.
  • Make sure to check-in with your students to see how many of their classes are also planning to do a take home or open-book exam as they tend to take more time to complete.

Tools

Stellar’s Homework feature enables instructors to compile and organize assignments in one location. Homework will automatically be added to the Materials page in Stellar. Students can download assignments and upload their submissions to the Homework page. Using LMOD, create assignments (homework), and collect and grade submissions using the Gradebook module. Dropbox’s free mobile app turns any smart phone into a document scanner. Students can scan homework and email it, or save to dropbox and upload to Stellar or LMOD. For documents, this produces far better quality than photos and also allows students to stitch all pages together into a single file for submission.

In addition to the options for grading and reviewing assignments discussed above, you may choose to use an external tool called Gradescope for assignment grading in conjunction with Stellar and Learning Modules.

Please note that Gradescope integration is not supported by IS&T at this time, but may be in the near future (updates to come). See here for how to move data in bulk between Stellar and Gradescope manually by exporting and importing spreadsheets or CSV files.

Case Studies at MIT

3.016 | Mathematics for Materials Science and Engineers

Professor Craig Carter 
Medium size class, materials science & engineering computational subject

Instead of a final exam, students will produce two videos:

  1. A pedagogical demonstration and visualization about the physics for a problem of the student’s choosing.
  2. A video with a tutorial of how the student obtained the solution and how their code works. Each video must be less than 5 minutes.

To help with this new assessment scheme:   

  • Students have been provided with more time to brainstorm their projects.
  • Instructional staff has provided more one-on-one interactions with students.

Professor Carter has also introduced a series of low-stakes assessments that he hopes motivates students: weekly physics/math/coding puzzles followed by a prize that students chose. Students receive a simple challenge (ex: find the average chord length of a circle).  Students who participate then get a choice of prize (e.g., pizza with instructional staff and co-participants at Area Four, ice cream party at Toscaninis’, chips and salsa party) to be given in the fall semester. “The last challenge was fun because two students got different answers and both were correct—an example of Bertrand’s paradox.  One student made a video of his solution to share,” reflects Professor Carter.

2.086 | Numerical Computation for Mechanical Engineers

Professor Dan Frey
Large class,  mechanical engineering computational subject

Professor Frey made the following modifications to this two summative assessments: 

  • Mid-term exam: a file will be posted on the course’s site which will be available for download when the exam period starts. Two hours later, students have to post their solution which is the file provided plus their answers added within the file. Given that the exam is not proctored, Professor Frey asks students to sign the following code of conduct: “I, FIRST NAME LAST NAME,  will not communicate during the exam period with any person by any means including voice, text, or file sharing.  I will not post any solution materials where they are accessible to other students during the exam period and for one hour after the exam (10:30 PM tonight).” The reason for including the last part of the statement is because several students in the course have received special time accommodations. 
  • Final project: the final project in the course is  based on coding, therefore plans for its administration and distribution have mostly stayed the same. The following was communicated to the students regarding exceptions for working in teams: “The final project will proceed mostly unchanged. An exception is that the Project Scoping document which was due Monday 16 March is now due Monday 30 March. Also, if any student wishes to switch from a team project to an individual project (because team coordination will probably be more challenging now), then just submit an individual project proposal on 30 March. Project coaching will be accomplished using individually scheduled appointments using Zoom, Skype, Facetime or whatever each project coach chooses. The project showcase will be canceled and only the written project reports will be required, but a virtual space for sharing these reports and for receiving comments from classmates will be arranged.”

2.007 | Design and Manufacturing I

Professor Sangbae Kim, Senior Lecturer Dawn Wendell, Professor Amos Winter + many coaches!
Large class, mechanical engineering design and manufacturing subject

Typically, this subject includes a major design-and-build project where students design and build robots that they showcase in a final competition. The students are not able to build their robots in a remote learning situation, so the syllabus and assignments have been revised to focus more on design.  

The final assessment will be a report detailing their design and analysis, and a presentation to their lab group about their final Robot design.  “We know that this changes exactly the details of what students will leave the class knowing, but we are confident that they will still be generally well-versed in design and manufacturing,” reflects Dr. Wendell.

 


1: Gurung, R. R., Wilhelm, T., & Filz, T. (2012). Optimizing Honor Codes for Online Exam Administration. Ethics & Behavior,22(2), 158–162. https://doi.org/10.1080/10508422.2011.641836