* Corresponding author
The COVID-19 pandemic led to dramatic changes in learning and teaching, and we had to change and innovate to deliver teaching and assessment online. As an institution we went from having a high proportion of invigilated exams, to moving these to open book and online. We had to amend the type of questions we used, using application and problem solving questions to assess our students, avoiding questions where students simply “Googled” an answer.
Given these significant changes, we were interested in what both students and staff in the Faculty of Science thought about the use of open book exams. This was in part to evaluate what we had done, but more importantly to identify if we should be retaining aspects of open book exams moving forward.
We ran a voluntary Google form survey for students in the Faculty of Science, and held student and staff focus group meetings. 308 students completed the survey (undergraduate levels 1 to 4). 188 students (61%) declared no health issue or learning difference. 33% (100 students) declared a health issue or learning difference. 36 students declared 2 or more health or learning conditions.
Positive messages
Open book exams allow students to demonstrate their ability, and assess real-world skills.
Open book exams provide flexibility and inclusivity.
Open book exams reduce stress.
135 and 111 students out of 308 either strongly agreed or agreed that open book exams allowed them to demonstrate their ability. Key to this was the fact that the exams were not a test of memory, but instead tested understanding and application. This was highlighted by both students and staff, with recognition that open book exams reflect a more authentic approach to assessment.
141 students had less stress, 109 the same and 58 more, in comparison to invigilated exams. Reasons for less stress were linked to having time to think, no need to memorise and being able to focus on understanding and not rote learning. Those with more stress highlighted they spent too long on exams and there was more pressure to perform.
There were 97 mentions of flexibility and 33 of open book exams supporting those with an SPLD or health issue. Interestingly, an analysis comparing those who had not declared versus those who had declared a disability / health issue showed that there was (for most questions and comments) no significant difference, i.e. open book exams allowed them to demonstrate their ability, there was less stress, longer exams were useful. Therefore, students across different groups identified the value of the open book exams in terms of flexibility and inclusivity.
Additional considerations
Plagiarism and collusion concerns.
More guidance for staff and students is needed.
The theme of collusion was very strong in the focus group discussions with both students and staff. For students, there was a feeling that those not engaging in collusion were being disadvantaged. Staff raised concerns around being able to detect and penalise for collusion. Another key point was around the line between what counts as cheating and what doesn’t. This all points to a need for clear and consistent guidance on what is allowed and what is not.
Only 39% of students knew what to expect from the exams, and a prominent point of discussion in the student focus groups was that of needing more guidance and support from departments.
Staff issues centred around a lack of consistency between staff in a given department and differences in response between departments. There were a number of references to the increased administrative burden that departments had to shoulder as a result of the exams being managed locally.
Recommendations
The wide-spread use of open book assessments has brought benefits to many students. These assessments are associated with reducing overall student stress levels, have greater relevance to future employment and are a more inclusive way of assessing students. We have challenges to overcome in ensuring that the assessments are robust and fair, but these problems should not push us to simply return to the exam hall. We recommend that departments should strongly consider keeping at least some assessments as open book (where appropriate), and that these decisions form part of a programme level approach to assessments.
View a more in depth analysis of the data (requires University of Sheffield login)
About the authors
Emma Jones is a senior university teacher in the School of Biosciences, with a strong interest in molecular biology and research led teaching. Louise Robson is a Professor of Digital Innovation in Learning and Teaching, who has championed the use of lecture capture and active learning.