The need for new approaches to evaluation in university courses
University teaching staff need to understand more about the experiences of students who return to university for postgraduate study and who are expected to learn by communicating in electronic discussion groups. Often these students have little chance of face-to-face interaction amongst themselves and with their tutors. They take part in courses using electronic communication either if studying in the UK or as students from other countries.
It is necessary to find the most effective ways that mature students use technologies for their learning. Currently, one way this is done is by asking students to record their levels of ‘satisfaction’ through post-course surveys, but this approach can be flawed. The problem is that it does not affect the quality of the experience at the time, and keeps evaluation of the experience separate from studying itself. It is important to develop opportunities for students to reflect on the learning processes they are going through in a way that makes a positive impact on their abilities to learn well in new contexts. This is one way of supporting students to develop the skills required to be a ‘lifelong learner’.
The experiences of e-learners
This case study describes two projects which focus on the experiences of e-learners on postgraduate courses. ‘A study of e-learners’ experiences in the mixed-mode professional degree programme, the Master of Teaching’ 2005-6
‘The development of methods for eliciting learner-narratives within a framework for embedded evaluation for fully online distance learners’ 2006-7
The work was carried out by a team based at the Centre for Excellence for Work-based Learning for Education Professionals. The team investigated the experiences of learners who are studying on courses that are either 100% online through a Virtual Learning Environment (VLE) or include a mixture of face-to-face and online learning. The focus was on students following postgraduate programmes in three UK universities and the results are expected to form the basis of recommendations for universities to adopt improved ways of evaluating learning experiences within courses which use VLEs.
Combining evaluation with e-learning
The approach the projects took was of continuous evaluation. Students are able to think and talk openly about their experiences of being e-learners throughout the learning process. The study shows that there are benefits for students in taking part in evaluation activities which encourage them to talk about and consider their experiences in a variety of ways – through one to one discussion, through online commentaries and in response to images which prompt reflection on e-learning processes. They are able to grow in confidence as e-learners and take greater control over the online discussions, deciding on the most appropriate ways to communicate and to gain the most from the learning activities.
E-learners’ experiences – what’s new for them?
The students identified four areas of experience which make a significant impact on how they adjust to learning with technologies on a new course. These areas affected students who were experienced e-learners at the start of a new course just as much as those who were e-learning novices:
- How to manage work, family or private life in flexible ways;
- Overcoming concerns about writing in an online forum;
- Adjusting to learning with other students collaboratively;
- Working out the social relationships with fellow students.
The team found that that there is little evidence that getting used to e-learning is a matter of steady progression. The most important gain for the students is in their own emerging self-awareness and understanding of the demands of learning in new contexts. They become more ‘expert’ learners, ready to move on to the next experience.
Most participants show considerable interest in how they learn with others and what impedes or supports this. They reflect on desirable features of online discussions and take on responsibility for the way the online discussions develop. Among some students there is increased commitment to working collaboratively (such as responding to online partners, sharing files and documents). For most, there was very little real understanding before starting of what becoming an e-learner would involve and how demanding that would be. Most predicted that flexibility and accessibility would be a real benefit, but few had anticipated the new ways they would be expected to learn with others in an online discussion group.
Tutors can gain a way of better understanding the realities of ‘being an e-learner’ on their courses. Finding time to evaluate learning experiences within the structure of a course can be a challenge, and demands a focus on the quality of the learning experience rather than easily measurable surveys of ‘satisfaction’. There is an argument for increased course time to be spent in developing the students’ abilities to reflect on how they are learning which can have a positive impact on what they learn.
Caroline Daly, project leader, concludes, saying “So many tutors take it for granted that their e-learners are having experiences which are positive and that they quickly adjust to the ‘newness’ of everything. They expect that the course is experienced in a coherent way – the way they planned it. But this is often not the case. We just don’t know enough yet about how people learn with technologies, and the only way we really begin to find out is by talking with the learners throughout the process”.
Funding
These projects were funded 2005-7 by The Centre for Distance Education, University of London and the Centre for Excellence in Work-based Learning for Education Professionals (WLE), Institute of Education, University of London.
Project team
Caroline Daly, WLE
Norbert Pachler, WLE
Jon Pickering, WLE
Jeff Bezemer, WLE
Jill Russell, The Open Learning Unit, University College London
Jon Wardle, Centre for Excellence in Media Practice, Bournemouth University
log in
