Descriptive account

Coping Strategies for Staff Involved in Assessment of Laboratory Write-Ups

Ian Hughes

Professor of Pharmacology Education, Learning and Teaching Support Network Centre for Bioscience, School of Biomedical Sciences, University of Leeds, Leeds, LS2 9JT, UK.

Date received: 28/11/2003             Date accepted: 1/4/2004

Laboratory-based practical exercises, which are an important and time-consuming part of many science degree courses, may be directed towards a variety of learning objectives. Some of these have traditionally been assessed by staff marking the student's written account of the laboratory experiment (the laboratory write-up) but increasing student numbers, which may have doubled or quadrupled on some modules, have made it difficult to sustain this approach. In addition, there is evidence that the formative element of the assessment (i.e. the comments written by staff on the laboratory write-up) is not fully utilized by students who are often only interested in the mark given. This paper reports on experience with the various strategies which may be used to cope with the increased marking load while maintaining or improving the learning gain from the formative element of the assessment. The adoption of a balanced mixture of strategies may present the best solution to the problem but must be tailored to local circumstances.

Introduction

Laboratory practical exercises are an important part of many science degree courses and may occupy 50% or more of the students time-tabled contact time with teaching staff. In the final year the proportion of time spent in laboratory work may be even higher. This considerable time involvement is associated with a variety of learning objectives and it is widely accepted that it is not just laboratory skills which are taught, acquired or practiced in laboratory classes (Hughes, 2001a). Some of the varied learning objectives (e.g. report writing skills, data handling and interpretation skills, knowledge and understanding of the principles involved and their application in a wider context) have traditionally been assessed by staff marking the student's account of their laboratory work, the laboratory write-up. This often involves an introduction (explaining the background and objectives of the class), an account of the methods used, the results obtained, their processing and interpretation and a discussion of the outcomes and the reasons for any deviations from the results expected. A number of questions may be set to be answered at the end of the write-up to test understanding and the students ability to make use of the information or principles involved in the practical exercise. The assessed element is often regarded as essential as students may choose not to attend laboratory classes or not to produce the required write-up if no marks are awarded.

The problem

The time spent by staff in marking a traditional laboratory write-up will depend on a number of factors but in first and second year classes may take 5 to 10 minutes or more. While this represents a reasonable marking load with a class or 18 students (3 hours per week) with a class of 200 the time involved is in excess of 33 hours (or more than 4 whole days) per week. Furthermore, the correction and feedback written on the laboratory write-up by staff is often not read by students who are concerned only with the mark awarded. This is clearly the case as students often make the same mistakes in succeeding practicals even when these are corrected in the feedback provided. Of the 23 students (out of 148) who made a particular error in one practical write-up 11 made the same error on a second occasion (1 did not hand in a write-up).  In addition, the inclusion of an instruction within the feedback to 'see the marker at once' frequently fails to elicit any response and when taxed about their failure to appear students claim to be unaware that they were required. Clearly the feedback is not always read, or if it is students fail to act on it.

These problems are most acute in first and second year where numbers of both students and practical classes are greatest. Over the years I have tried a variety of strategies to cope with the marking load while maintaining or preferably improving the learning gain from the practicals and the utilisation of the feedback provided to the students on the strengths and weaknesses of their laboratory report. These have included:

·         Work in groups and require a joint rather than individual report;

·         Run parallel non-laboratory activities on some weeks;

·         Abolish half or more of the practicals and provide time for self-directed learning;

·         Mark only some of the laboratory exercises;

·         Get your post-graduate students to do the marking for you;

·         Specify a very tight format for the write-up confined to reporting of the data recorded, its processing and the final result obtained;

·         Use MCQ and EMSQ (Wood, 2003) to test understanding of the practical /calculations;

·         Use pre-prepared comment sheets to provide feedback or use electronically assembled feedback through a managed learning environment;

·         Use peer- or self- marking of the write-ups;

·         Offer practically intensive and practically non-intensive options;

·         Separate the learning objectives of practical classes and teach each explicitly.

·         Each of these has advantages and disadvantages which are outlined below.

Possible strategies - their advantages and disadvantages

Work in groups and require a joint rather than individual report.

Rather than students working individually, grouping can take place for the whole or just for part of the laboratory exercise and the write-up. Students can be required to perform the experiment individually and then produce a joint write up with a partner. Equally, a pair of students can perform the experiment and produce a joint write-up (reduces marking load by 50%) or can join with another pair to produce a single write-up (reduces marking load by 75%). The latter option involves combining two sets of data and therefore gives experience of this essential aspect of data handling (since in biology it would be rare for an experiment to be performed once only and presenting data from combined repetitions of an experiment would be normal).

There is some evidence that grouping produces an improvement in the quality of the write-up. All students in a first year class performed the experiment in pairs and some then produced an individual write-up (mean mark 58±6%; mean±standard error (m±s.e.); n=64) while others produced a joint write-up from the work of two pairs (mean mark 74±4%; m±s.e. n=8). These two mean marks are statistically significantly different (P<0.001; t-test) but these unpublished results are not robust as the students were self-selected and may have been a more able sub-group of the class. In addition the tasks were different (since data had to be combined and the marking criteria were not identical). The students involved in producing the joint write-ups enjoyed the process however and of the 32 students only 2 did not wish to write up other experiments as a group (both cited the difficulty and inconvenience of arranging group meetings as the reason).

Monitoring individual progress is difficult if group write-ups are used exclusively and assessment of jointly produced work is of course not without its difficulties Acknowledging differential contributions is a well known problem to which there is a variety of partial solutions (Rust, 2001).

This strategy does not address the need to improve the utilization of feedback by students though the improved mark obtained by those working in groups does suggest some peer teaching has taken place with possible improved learning.

Run parallel non-laboratory activities on some weeks.

This strategy reduces the marking load by replacing some of the practicals with other activities which generate less marking load. The recognition of the importance of students possessing a variety of skills (e.g. information retrieval, poster presentation) makes this a particularly appropriate strategy. Literature searching exercises, often web-based, and data handling exercises also make suitable alternatives as does training in verbal communication skills which can be self- or peer- assessed (Hughes and Large, 1993). MCQ/EMSQ  tests (computer marked) can  be substituted for some practicals one and two thirds through the semester which then provides staff and students with a monitor on student learning. This does of course cut down the amount of practical experience the students obtain though other important non-laboratory skills may be acquired instead. Again, this strategy does not address the utilization of feedback issue.

Abolish half or more of the practicals and provide time for self-directed learning.

This strategy reduces the marking load and is often popular with some students since they are required to do less work. However, biology is essentially a practical subject and even those students who are progressing to non-laboratory type jobs need to have an appreciation of laboratory work since the experimental method and design employed can affect the result obtained and may be crucial to the conclusions which can be drawn from a piece of work. If the balance between practical classes and other types of teaching methods is already correct on the module then the use of this strategy must raise questions about reduced quality.

Mark only some of the laboratory exercises.

Traditionally, every practical session will result in a write-up and all will be marked. This is in part to provide the 'stick' to ensure that students not only attend the practical session but also complete the write-up during the production of which much of the learning takes place. It is possible to mark write-ups associated with only some of the practical classes but if students are aware which write-ups will be marked and which will not then many students will make little or no effort at those which are not to be assessed. If all write-ups are required to be handed in and students are only told if the write-up is to be formally assessed after the deadline they must therefore make maximum effort on each or run the risk of loosing marks. The problem that no feedback is provided on those write-ups not marked can be overcome if a separate generic feed-back session is arranged for all the students together. Care must be taken how this is timetabled however or students will deduce which write-ups are to be marked and which not. This information is also passed from year to year so changes must be made in which write-ups are marked in succeeding years. A generic feedback session after every practical works well and would be expected to enhance the learning from each practical.

Get your post-graduate students to do the marking for you.

This is a frequently adopted strategy and is clearly successful in reducing the marking load on the member of academic staff. Post graduates need appropriate training in assessment however and must be provided with a clear marking schedule. If more than one marker is used it is wise to test that the mean mark given is not significantly or consistently different and that equivalent feed-back is being provided by all markers. There may be some reluctance by academic staff to allow their post-graduates to spend time in this way. However, there may be more emphasis on providing training of postgraduates in supporting the learning of others as part of improving the standards in postgraduate research degree programmes (Higher Education Bodies, 2003). Again the strategy does not address the utilization of feedback issue.

Specify a very tight format for the write-up confined to reporting the data recorded, its processing and the final result obtained.

Marking of practical write-ups can be significantly speeded if the reports are short and focused on the work done in the laboratory. This can be achieved by providing a set format within which students must produce their write up. For example in a workbook which has empty tables (in which the students fill in the numbers), the axes of graphs (on which the axes labels and points must be inserted) and proforma setting out the skeleton of the data handling methods. Marking time can be reduced by more than 70%. This sort of report can even be written up, handed in and marked as part of the class (Gibbs et al, 1993). There are disadvantages however in that all initiative is removed, report writing skills are not practiced and completed workbooks are handed down year to year.

Use MCQ or EMSQ to test understanding of the practical/calculations.

This involves students writing up the practical in the normal way and handing in the write-ups. In the following practical the first 15 minutes is spent in answering say 8 MCQ or EMSQ which are computer marked and are designed to test understanding of underlying theory, the correctness of the values obtained or derived and the method of calculation used. These questions can be marked electronically and immediate correctional feedback provided in printed form (Booth & Wood, 1989). This strategy reduces the marking load very successfully and maintains the level of practical experience. Development of report writing skills is not achieved however since the write-ups are not assessed.

Use pre-prepared comment sheets to provide feedback or use electronically assembled feedback through a managed learning environment.

When marking practical write-ups it soon becomes clear that many students make the same mistakes or omissions in their write-ups and the same comments are therefore written on many write-ups. The marking process can be significantly speeded by producing a printed sheet specifying and correcting common (numbered) errors and omissions. All the marker has to do therefore is to write the designated number corresponding to the required feedback on the write-up and return it with the printed sheet so the student has a full explanation. This can cut the time required for marking by more than 50% and does provide clear and readable feedback to the student since the sheet can be carefully prepared to give a full explanation. The sheets do get handed down from year to year and this can be a problem as the students simply copy the material from the feedback sheets. The extent to which this occurred in one of my classes was very considerable and forced me to develop a different experiment to be used in successive years. A less serious problem was to get the students to understand that the numbers in the margin referred to the numbered comments on the feedback sheet and were not in fact marks awarded!

Software systems used in managed learning environments (MLE) may allow a similar process to be carried out electronically on electronic or paper-based submissions. Pre-prepared feedback items can be selected as appropriate for each practical and assembled into an individualised feedback report which can be sent electronically to the students (Pitts & Bolton, 2002). This does mean of course that marking has to be done when the MLE is accessible (not on the train for example). Again the utilization of feedback issue is not directly addressed although there is anecdotal evidence that the electronically delivered individualized feedback report is read by students.

Use peer- or self- marking of the write-ups.             

This strategy, reviewed recently by Race (2001), enables the marking of 200+ practical write-ups in less than 1 hour and is therefore extremely successful in reducing the staff marking load. It works best for 1^st or 2nd year practicals as the key is the use of an explicit marking schedule which all the students follow. This is easiest to write when the experiment involves following a proscribed schedule, the collected data is largely similar for each student and each write-up follows a closely formatted pattern. Under these conditions marking by students is both accurate and reproducible. Students must pay attention to what should have been written as they have to assess the work of others (peer-marking is preferred to self- marking as the opportunities for cheating are reduced) and therefore each gets a full explanation of what they should have done. In addition there is a bonus in that students come to grips with the problems of assessing the work of others, a skill they will require surprisingly soon after taking first employment. There is evidence that peer marking improves the standard of subsequent practical write-ups (Hughes, 1995; 2001b).

Offer practically intensive and practically non-intensive options.

Thirty years ago the majority of bioscience graduates obtained employment in bioscience most often in laboratory situations. Consideration of destinations of pharmacology graduates and HESA data (Hughes et al., 1997; http://www.bio.ltsn.ac.uk/issues/employability/) indicates that less than half the bioscience graduates now take employment involving bioscience areas and even fewer are employed in contexts which use bioscience laboratory skills. Students are heterogeneous in their needs for laboratory skills training and it is possible to offer core modules which contain relatively little laboratory work and optional modules, for those who may regularly need bioscience laboratory skills, with a very high laboratory class content. This has the advantage of targeting scarce laboratory resources towards those who will need them and reduces the numbers in the laboratory class. Many students, while being uncertain of the particular employment to which they aspire, may have decided relatively early, sometimes even before coming to university, that a career in laboratory work is not for them. Only those students who are highly motivated towards laboratory work choose to take the laboratory intensive modules and this can be documented in a personal portfolio.

Separate the learning objectives of practical classes and teach each explicitly.

Reference has already been made to the variety of learning objective which can be associated with a laboratory class and it should be noted that the traditional laboratory write-up is not a particularly effective way of assessing actual laboratory skills (i.e. performance in the practical manipulations required in the laboratory). These can be assessed explicitly and separately from the laboratory report. Similarly, writing say 10 essentially similar reports of laboratory experiments during a semester is not necessarily the best way to teach report writing skills. Again these can be taught appropriately and assessed separately as can each of the other learning objectives. Thus data handling, data interpretation, data presentation, report writing, experimental design etc can each be taught and assessed in the most appropriate manner. This strategy may eliminate the need for the production of the traditional laboratory report though it will generate other items which have to be marked by staff or in some other way. The feed back provided on each of the exercises can be tailored to be most effective. 

Conclusion

It should not be thought that the solution to the problem of assessing large numbers of laboratory write-ups and ensuring the effectiveness of the feedback provided will necessarily be found in any ONE of the above strategies. They are not intended to be exclusive and the best solution may involve the use of several of the strategies within a single module. It should also be appreciated that the usability and the effectiveness of the strategies will depend on the emphasis placed on each of the various learning objectives and on local circumstances. In units where laboratory classes are organised on a 'circus' basis because of lack of space or equipment the use of peer marking may be unrealistic as the first group to be given an explicit marking schedule are likely to pass this material to those yet to do the practical. The advantages and disadvantages of each strategy must therefore be considered in the light of the local circumstances. However, careful consideration and use of these strategies should enable teaching units to maintain or enhance the quality of the provision even in the face of sharply rising numbers, falling resource and competing demands on academic staff time.

Communicating author Ian Hughes, Professor of Pharmacology Education, Learning and Teaching Support Network Centre for Bioscience, School of Biomedical Sciences, University of Leeds, Leeds, LS2 9JT, UK: Tel 0113-343-4313; Fax  0113-343-4228; i.e.hughes@leeds.ac.uk

References

Booth, A.G.and Wood, E.J. (1989). The microcomputer in biochemical education. Progress in Medicinal Chemistry, 26, 323 - 354.

Gibbs, G., Habeshaw, S. and Habeshaw, T. (1993). 53 interesting ways to assess your student. Bristol. Technical and Educational Services. 

Higher Education Funding Bodies. (2003). Improving standards in postgraduate degree programmes. (May 2003.23). http://www.hefce.ac.uk/Pubs/hefce/2003/03_23.htm
Accessed 14th November 2003

Hughes, I.E. (1995). Peer assessment of student practical reports and its influence on learning and skill acquisition. Capability, 1, 39 - 43.

Hughes, I.E., Hollingsworth, M., Jones, S.J. and Markham, T. (1997). Knowledge and skills needs of pharmacology graduates in first employment: how do pharmacology courses measure up? Trends in Pharmacological Sciences, 18, 111 - 116.

Hughes, I.E.. (2001a). Alternatives to laboratory practicals - do they meet the needs? Innovations in Education and Training International, 38, (1) 3 - 7. 

Hughes, I.E. (2001b). But isn't this what you're paid for? The pros and cons of peer and self assessment. Planet (the LTSN Centre for Geography, Earth and Environmental Sciences Bulletin), pp 20-23,  June 2001.

Hughes, I.E. and Large, B.J. (1993). Assessment of students' oral communication skills by staff and peer groups. New Academic, 2, 10 - 12.

Pitts, C. and Bolton, M. (2002). Automating feedback to students. http://www.ldu.leeds.ac.uk/ldu/projects/cit/cit7.htm Accessed 14th November 2003

Race, P. (2001). A briefing on self, peer and group assessment. LTSN generic centre assessment guides series No. 9. http://www.ltsn.ac.uk/genericcentre/

Rust, C. (2001). A Briefing on Assessment of large groups. LTSN generic centre assessment guides series No. 12. http://www.ltsn.ac.uk/genericcentre/

Wood, E.J. (2003). What are Extended Matching Sets Questions? BEE-j Vol 1, Issue 1 http://bio.ltsn.ac.uk/journal/vol1/beej-1-2.htm
Accessed 14th November 2003

PDF version

A PDF version of this article is available