The National Postgraduate Committee (NPC) has received a number of enquiries in recent days regarding institutional audits run by the Quality Assurance Agency (QAA) for which it hopes this publication will answer several frequently asked questions. This short document intends to give some brief guidelines from the postgraduate perspective on how postgraduates can be included in the audit process as well as how their circumstances are different to that of undergraduates. Specific attention has been given to the new code of practice for research degree programmes produced by QAA to explain from the student perspective what significance the code has as well as being able to use it for the auditing of research degrees. It is not the intention that this publication should tell student representative bodies how to write a submission for postgraduates in their audit or that it should re-interpret the code of practice. It is rather the intention to give a starting block from which to investigate further and produce something original to the institution from the postgraduate students' perspective. Several other existing publications are referenced in this publication as a guide to further relevant reading.
2 Specific points about postgraduates in auditing
The main guidelines for producing a student submission in an institutional audit are presented in Annex D of the QAA's handbook on institutional audits  and the QAA's guide for student representatives . In this document some important quotes with respect to postgraduates are worth noting as follows:
"5...students should feel free to provide whatever information they feel is appropriate (providing that it is relevant to the focuses of the audit) and to organise it as they choose."
First and foremost, the content structure should be still as flexible as possible and this publication does not intend whatsoever to prescribe a specific content for the written submission. It is important that an appropriate submission suitable to the institution and the student representative body (or postgraduate representative body if applicable) is to be included. Past experience has shown, however, that some submissions have easily included irrelevant information (i.e. information that is not answerable to the focuses of the audit). For example some submissions have included aspects of social life and also other institutional services not in the context of the student's learning experience. These points are inappropriate to submit to the audit since they are beyond the scope of what QAA exists to monitor. It is important that the main focus of the audit is considered carefully from the student perspective and that it looks at the institution's practices on a general level. These main focuses of the audit are indicated in the handbook as follows in Annex D  and the QAA's guide for student representatives :
- the accuracy, completeness and reliability of the information published by the institution about the quality of its programmes and the standards of its awards (this might include the accuracy of publicity materials and the use made of programme specifications);
- the information that students receive about the academic performance expected of them, their experience of the challenges posed by their programmes of study, and the ways in which their performance is assessed (this might include matters such as the usefulness of programme handbooks, aspects of assessment arrangements, and the feedback that students receive on their academic performance);
- the experience of students as learners (this might include the quality of academic and non-academic support, and access to learning facilities); the opportunity for students to participate in the management of quality and standards within the institution (this might include opportunities for committee representation at institutional and programme level, and other means of providing feedback to staff)."
The three above points on the publicity of programmes, guidance on standards and the learning experience are of vital importance to include in the written submission in a way that the student body sees as appropriate so as to provide their perspective of the institution in their own words. This guide intends to give ideas as to how postgraduates should be considered specifically in this regard, how they can be included in the auditing process and also how the new code of practice for research degrees  (published by the QAA in 2004 with much involvement from the NPC) will be applicable in the audit process.
The most important point in an annex of the handbook  where postgraduates are specifically referred to is as follows:
"6...If, for example, the submission has been prepared entirely from the perspective of undergraduate students or full-time students, then this should be made clear."
The fact that this statement is included does indicate the possibility that less represented students such as postgraduates, part time and franchise students could easily be left out, which is often due to inherent difficulties in getting student feedback from those who have less direct contact with their representative body. It is also not possible to submit two separate submissions so all students including postgraduates should be included in one single submission and if some particular students have not been included or taken part for whatever reason, this should be made clear.
The following two subsections will therefore indicate what specific features about taught postgraduates should be considered. The next main section covers the code of practice for research degrees, which will help to identify how the more specific interests of research postgraduates can be considered and in what way they can be accounted for by considering the standards set by the code. Finally the last section addresses the problem of postgraduates being an under represented group and gives useful references to how this involvement can be improved and applied to institutional audits.
2.1 Specific issues for Taught Postgraduates
The views and expectations of taught postgraduates are going to differ widely to those of undergraduates taught at a less advanced level. To begin with a vast number of taught postgraduates are significantly older, probably with some professional experience, studying part time or international. This will possibly create a significant difference in the nature of responses to certain aspects of the audit, which may warrant a separate questionnaire or methods of collecting data. When collecting data, it is important to consider whether such groups of students will relate to what is being asked of them. It is important to consider in this regard that audit is being taken in context not only for the average undergraduate who may have recently left school or college but also for students like postgraduates coming into or back to higher education with significantly different previous experience.
The diverse sources of funding and variety of courses for taught postgraduates will result in there being a wide variety of views and levels of expectation. This range will vary from sponsored students on a part time programme possibly not going to too much expenditure through to self funding students in business schools paying excessively higher fees than the majority of courses in other disciplines. In the undergraduate case, there is a more general structure of funding and costs are more consistent for home and overseas students. Further to this, there are many part time or vocational based postgraduate courses.
The following points have been included below to give an idea of what specific issues regarding taught postgraduates could be considered. This list is by no means exhaustive and it only has the intention to help individual representative bodies generate appropriate ideas for themselves. Many of these ideas may also be applicable to undergraduates in a similar way.
- Are there clear indications about qualification requirements? Is there a course handbook for example, appropriate induction sessions or other important briefing information to raise awareness?
- Are the modules/contents clearly mapped out in the courses as to what exactly is included in the course or required of the student to gain a qualification?
- How much further beyond the undergraduate qualification might it go, can a prospective student identify clearly whether the course they are interested in will give them significant extra qualifications necessary on top of what they already have?
- Part time study implications. Learning, understanding and communication are all but a few of the very different issues that part time students face. It is very important to look at the many courses that are intensively part time to see how their delivery will have some specific differences.
- Supporting a wide diversity of students' abilities. For the varying levels of both national and international professional experiences postgraduates could have, this scale of diversity is much wider than that of undergraduates. Do courses accommodate this and allow each student to develop further from whatever level they are at?
- Specific development at postgraduate level. Again, their level of current attainment may mean they may require some more appropriately tailored or advanced personal development and learning support opportunities that would not apply to undergraduates.
- Established liaison committees. Do these actively involve postgraduates sufficiently for both part time, full time and others who may need specific representation?
- Are there evaluation procedures in place to proactively identify room for improvement and then report on how such improvements have been approached?
3 The Code of Practice for Research Degrees - what is it about?
An important aspect of work within the higher education sector for the NPC, especially during 2004 was its involvement in the development of the QAA Code of Practice for Research Degrees . The importance of this document at the time of writing has a significant impact in England, Wales and Northern Ireland because institutions who are not in alignment with the code could face reduction or loss of funding for research degrees. To meet the required standards in the code, it is planned that the current institutional audits will include postgraduate research degree programmes from 2006. Before then it is planned that there will be a special review of the quality of research degree programmes carried out by the QAA across the UK , which will help to shape the audit. However, as this develops there are important aspects of the code that postgraduate representatives should consider in order that the use of the code has transparency in how well research degree programmes are delivered to the student. At the time of writing no plans have yet been brought forward as to how research degrees will be considered in Scotland , who have a different enhancement-led review system though the code is still applicable across the UK.
Within the code there are 9 sets of precepts, and these have been described in terms of how they are of interest to postgraduate research students. It is advised that the code of practice itself is read carefully while reading this summary also. It is important to emphasise at this point that what is written here should not be used to as a direct interpretation of the code of practice. All representatives are encouraged to read the code directly  and then consider how different aspects could be best implemented within their institution. The purpose of this section is to assist in understanding and realising the value of the code so that postgraduate representatives can identify what standards institutions should be implementing in delivering their research programmes.
Institutional arrangements (Precepts 1 to 4): These introductory precepts are to ensure that institutions will develop or redevelop their own code of practice for research degrees. Such a code will be required to be an openly available and effective document that can be used to ensure postgraduate research programmes are being properly delivered. From the student perspective, representatives should be encouraged to read through the code of practice and comment on how clear it is to the student in understanding what they are required to do in their research degree programme. They should also consider commenting on how they see the code being implemented and actually demonstrating a difference at the receiving end.
Research environment (Precept 5): One important thing about undertaking a research degree is having access to a cognate group of researchers in the area. This can be difficult in some small institutions, or indeed in some very specialist subject areas where each researcher in a department can specialise in a significantly different area to that of other researchers. Therefore accessing a wider view on a research topic can have complications. In some subjects this is not so much a problem, where there are a wide range of researchers to consult. Where this is not the case, access to appropriate researchers in other locations could be required and so some institutions or departments will need to ensure this. An important factor for the student in this regard is whether the institution provides means for the student to interact with others through events such as seminars as well as having access to publications, equipment and a place to meet so that they can have the social space they need. It is also worth noting under this precept the need to have effective student representation, possible advice on career development and welfare support for postgraduates.
Selection, admission and induction of students (Precepts 6-10): It is important to ask whether the admissions process is secure and whether the supervisors are accepting students onto a research programme without enough due care and attention to see if they have what it takes. These precepts put guards in place to ensure such ad-hoc admissions are avoided. Therefore the student should consider if admissions are being conducted fairly for applicants and if they are being given the correct advice. There are also guidelines on the induction process, which NPC was involved in writing, in terms of how well a student is being introduced to their programme. These include inductions at institutional, departmental and supervisor to student level. Therefore are these induction programmes helpful and do they ensure that the student has been given an appropriate introduction to enable them to begin their programme? Most importantly have they been able to establish an appropriate working relationship with their supervisor and be introduced properly to all the facilities they need to use?
Supervision (Precepts 11-14): This is one of the most controversial parts of the code in terms of how there is provision for transparency in the supervision process to see if the student is receiving inadequate or inappropriate supervision that could lead to severe difficulties later on. Whether to have more than one supervisor, a separate tutor or some other arrangements is a difficult question to answer and it will apply differently between disciplines often. Further to this, supervisors are required to have the expertise in the subject area and not be overloaded with other responsibilities in order to properly supervise. It is important to read all aspects of these precepts in the code to identify what problems and difficulties there are in supervision and whether there is clear guidance in what role the student and supervisor have.
Progress review (Precepts 14-17): It is important to review progress regularly and plan ahead. Not only this, but it is important to look over how a student is developing in their research and other skills. Therefore is the student receiving this so that they have the help they require or is the whole process too bureaucratic and wasting too much time that it has no worth? Some institutions have very successful progress review methods for this that help the student and supervisor to quickly see where things are going well or going wrong. However, some students and supervisors avoid the process, without identifying its importance to ensure the student is being appropriately supervised and making progress as well as identifying their training needs.
Development of research and other skills (Precepts 18-20): This is a wide subject to deliver training to a wide diversity of students, from recent graduates seeking career development through to already experienced professionals and even retired individuals that have far different training needs. Therefore, if all are delivered the same training, many are going to be frustrated. The diversity and culture of students needs to be considered carefully in order to deliver the appropriate training to develop each individual. Therefore representatives may consider what discipline areas the institution covers and what the needs are of those students and whether the training is appropriately tailored and able to meet their needs.
Feedback mechanisms (Precept 21): Postgraduate representation is important, and especially for postgraduates as NPC has always promoted. This code supports that all the more and it should enforce institutions to help in the process, where by there are effective feedback mechanisms, both local and central through which appropriate action can be taken on issues raised by postgraduate research students. This is another area where student representative bodies have a responsibility to take part and avail themselves to postgraduates as an independent body that can extend its remit to postgraduates. Therefore it should be questioned how effective any feedback systems are and if there is open opportunity and support for a representative group of postgraduates to be involved in representing collective issues.
Assessment (Precepts 22-24): All doctoral programmes have to be assessed normally with a viva examination at the end. How the viva is conducted does vary across institutions. The code raises the question as to whether an independent chair in a viva is useful in terms of giving an impartial view as to the fairness of the examination when a student has failed or is asked to resubmit. Such problems can cause immense difficulties in appeal cases if there were no witnesses to see what went wrong in the examination and whether any mitigating circumstances could have affected the outcome. Representatives therefore should consider if the assessment of research degrees has a transparent system. Further to this are the assessment procedures made clear to the student so they are aware of what they need to do – could any less reliable communication cause difficulty? One may find very few research students able to answer when questioned about what criteria they need to meet to pass their research degree examination and successfully defend their work.
Student Representations (Precepts 25-27): Not all students find supervisors to be fulfilling their role properly and some supervisors may even go against it. Sometimes supervisors can be very domineering or cause great problems for the student that is impeding their progress and leading them to extremely difficult circumstances. Any student should proactively complain when they need to or be able to freely appeal should an examination or review of progress be unfair. It is very difficult if in the first instance if the student has to complain to the academic next door who is close friends with the supervisor. Such a situation can exist and it is obviously not impartial to be able to help the student deal with their problems. Student representatives should critically consider whether they know what complaints procedures there are, if they are effective, impartial and dealt with the due care necessary. Understanding of the appeals process and their difference from complaints should also be clearly explained.
4 Involving postgraduates in the audit - how is it different?
Involving postgraduates in the audit process is as hard as any other aspects of involving postgraduates in support and representation. There is little need to go into much detail in this publication as to how postgraduates need to be involved in the processes but some examples of good practice will be cited. It is worth noting that a number of sections, most notably the following, contained in the resource folder of the NPC , will be of great use. These resources are available freely online to affiliates of the NPC at http://www.npc.org.uk/essentials.
- How do we involve postgraduates? - The Answers
- Representing Postgraduates in Small Institutions
- Postgraduate Based Institutions
- Communicating to Postgraduates
- Representation of Part Time Postgraduates
Considering the more specific aspects of involving postgraduates in an audit process, some of the following tips will be of benefit in this regard.
- If you do have a questionnaire, would a separate one be appropriate for taught postgraduates and do consider how would the very different aspects of research postgraduates be included in one single questionnaire with undergraduates also?
- Could you also include questions not relevant to the audit, thus you could "kill two or more birds with one stone". Another benefit of this is that it will possibly include questions that students may more passionately want to respond to so that they will give more effort towards the questions that are relevant. How questions are worded matters significantly to the information gathering and should be carefully considered in this regard.
- Is there a specific postgraduate committee and is their role sufficiently involved in the process to directly represent the interests of postgraduates to the running of the information gathering? Such factors are of vital importance.
- How would you communicate to postgraduates? This is not as straight forward as it may be for undergraduates via posters, student events and other things that are largely used by undergraduates. Postgraduates are in fragmented pockets of an institution, with few central places that they congregate so information in the NPC resource folder will assist here.
- Considering the differences of postgraduates to that of undergraduates is very important, they are a distinctively different layer of community in a higher education institution and have to be regarded differently. Again, the information presented in the resource folder from NPC on how to involve postgraduates, have appropriate representation structures and other information will help in this process.
- Is an institution largely or entirely postgraduate? If this is the case, then from the start a completely different context needs to be taken on board. Also it is wise to consider the diversity of those postgraduates as to how many are research, taught or on other non higher degree level programmes. Ensuring inclusion and balanced involvement in these regards is still essential.
This brief information on involving postgraduates is there to help start off the process of involving postgraduates, which may extend to many other aspects of postgraduates within a student representative body. For further information of any kind both related to auditing issues and anything else, the NPC's contact details can be found at http://www.npc.org.uk/contact.
5 References and further reading
- Institutional audits handbook, Quality Assurance Agency, 2002, http://www.qaa.ac.uk/reviews/institutionalAudit/handbook/audit_handbook.asp.
- Code of Practice for the Assurance of Academic Quality and Standards in Higher Education, Section 1: Postgraduate research programmes, Quality Assurance Agency, September 2004, http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/section1/default.asp.
- T. Brown, E. Gillon, J. Hoad, Resource Folder, National Postgraduate Committee, 2004, Second Edition, http://www.npc.org.uk/essentials.
- Consultation with the National Postgraduate Committee, http://www.npc.org.uk/contact.
- Operational description for the special review of postgraduate research degree programmes (England and Northern Ireland), Quality Assurance Agency, 2005, http://www.qaa.ac.uk/reviews/postgraduate/postgraduateengland.asp
- Code of Practice for the assurance and standards of academic quality and standards in higher education, Quality Assurance Agency, http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/default.asp
- Institutional audit: a guide for student representatives, Quality Assurance Agency, 2003,http://www.qaa.ac.uk/students/guides/instauditguide.pdf
- The framework for higher education qualifications in England, Wales and Northern Ireland, Quality Assurance Agency, 2001, http://www.qaa.ac.uk/academicinfrastructure/FHEQ/EWNI/default.asp
- Enhancement-Led Institutional Review: Scotland, Quality Assurance Agency, http://www.qaa.ac.uk/reviews/ELIR/default.asp.
- NPC Guidelines on the Codes of Practice for Postgraduate Research, National Postgraduate Committee, 1992, http://www.npc.org.uk/page/1003801720
- NPC Guidelines on the Codes of Practice for Instructional Postgraduate Courses, National Postgraduate Committee, 1993, http://www.npc.org.uk/page/1003801941
- NPC Guidelines on the Conduct of Research Degree Appeals, National Postgraduate Committee, 1995, http://www.npc.org.uk/page/1003802150
- D. Staniford, T. Brown, Complaints in Practice: Complaints in Crisis, National Postgraduate Committee, 2003, http://www.npc.org.uk/page/1074001547.pdf.
- A national review of emerging practice on the use of Personal Development Planning for postgraduate researchers, UKGRAD, Centre for Recording Achievement, National Postgraduate Committee, 2004, http://www.npc.org.uk/page/1098797810.pdf
- T. Brown, National Survey on User Perceptions of Personal Development Planning for Postgraduate Research Students, National Postgraduate Committee, 2005, http://www.npc.org.uk/page/1116239142.pdf