The NPC is the primary representative body of postgraduate researchers and students in the United Kingdom. Our affiliate membership comprises of democratically-elected student Representative bodies at Higher Education Institutes (HEIs) throughout the UK. As a registered charity (SCO33368), our aim is the promotion of postgraduate education in the United Kingdom. In order to achieve this aim, we host and address conferences, share best practice through publications and meetings, respond to consultations and campaign on issues of concern to postgraduates. We also co-operate with other democratic student bodies and trades unions who share our aims.
The National Postgraduate Committee welcome the opportunity to comment on the proposed changes to research assessment and funding by commenting on the specific questions in the DfES document "Reform of Higher Education Research and Assessment and Funding: A Consultation Document".
1 Which, if any, of the RAE 2008 panels might adopt a greater or wholly metrics-based approach?
The National Postgraduate Committee believes existing criteria should be applied based on the Government´s decision that RAE 2008 will proceed in principle. The NPC believes that all subjects should remain largely peer reviewed, adopting a minimum of metrics-based assessment.
The National Postgraduate Committee believes that the most important concern is whether the outcome of RAE 2008 and any shadow metric exercise will be allowed to determine funding.
2 Have we identified all the important metrics? Bearing in mind the need to avoid increasing the overall burden of data collection on institutions, are there other indicators that we should consider?
The National Postgraduate Committee believes that all the important metrics have not been identified. We believe that there should be a single coherent framework that encompasses research quality assessment and research funding. QR funding which supports infrastructure, research capacity and staff costs should encompass quality assessment metrics.
Metrics required for quality assessment should include some form of peer review, a value added metric, such as total research income as ratio of QR income and an output metric.
Any metric-based approach should be concerned with inputs and outputs.
Metrics for basic research infrastructure should consider the number of PGR students and number of staff on contracts requiring them to do research, the mix of subject areas and output factors such as publications benchmarked against world averages.
3 Which of the alternative models described in this chapter do you consider to be the most suitable for STEM subjects? Are there alternative models or refinements of these models that you would want to propose?
The National Postgraduate Committee does not believe any of the models would be suitable as they rely on a single metric. The NPC believes that while science and technology subjects might benefit from metrics based approach, all subjects should remain largely peer reviewed and adopt a minimum of metrics-based assessment.
The focus on STEM subjects also implies a separation from other subjects and we believe that any changes to the funding system should be fairly balanced to cover all subjects; a single system should allow discipline specific modifications. This would be necessary to prevent unfair funding distribution and to all new cross-disciplinary research
4 What, in your view, would be an appropriate and workable basis for assessing and funding research in non-STEM subjects?
The National Postgraduate Committee believes that all subjects should remain largely peer reviewed and adopt a minimum of metrics-based assessment. We recognise that there exists a limit amount of metrics-based assessment and believe that any changes to the system should create a system that works for all subjects and disciplines and maintains a limited level of metrics-based assessment. We believe that any changes to metrics in the non-STEM subjects should be fairer and focus on outputs and inputs alongside peer review.
The use of a standard system for all subjects, with a form of metrics in a common framework that can have subject grouping variation would be a minimum assessment alongside a broad panel of experts that would act as a peer review and monitor of subject metrics.
5 What are the possible undesirable behavioural consequences of the different models and how might the effects be mitigated?
The National Postgraduate Committee notes several concerns based on changes to the funding system proposed by the consultation. We believe the models are biased against new researchers as departments seek to build experienced researchers for metric output. We further feel that the focus of the changes should encompass training and development of researchers to ensure quality research is taking place and provide opportunities for new researchers. Moves to increase equal opportunities must be maintained and any move by metrics to increase the attractiveness of a `transfer market´ must be prevented.
The models are also biased towards established centres and against new research subject areas. The proposed system´s change of focus on output measures to input measures instead is concerning in that it will destabilise the system by introducing a lack of clarity and reduce quality and quantity of the research output.
The dissemination of information is important and the proposals do not go far enough to encourage the use of textbooks and other publications for dissemination. The focus on metrics may also further disadvantage teaching based institutions focussing less on publications.
6 In principle, do you believe that a metrics-based approach for assessment or funding can be used across all institutions?
The National Postgraduate Committee believe that if the system takes account of our concerns in our responses Qs 2-5 and ensures a strong basis of peer-review then there is no reason against this approach being used across all institutions. We do feel however that there is a move to consider metrics afresh recognising the concerns raised in this response.
7 Should the funding bodies receive and consider institutions' research plans as part of the assessment process?
The National Postgraduate Committee believes that funding bodies should receive institutions´ research plans but that these should form only a part of the metrics base-line funding.
8 How important do you feel it is for there to continue to be an independent assessment of UK higher education research quality for benchmarking purposes? Are there other ways in which this could be accomplished?
The National Postgraduate Committee believes that benchmarking is positive tool and should be reflected in research funding but that benchmarking should be a separate function to the allocation of all research funding.
Benchmarking takes place in the public sector to assess and evaluate performance but is rarely used as a sole determinant of funding. The opportunity for benchmarking is for capacity building to encourage funding as a tool alongside other criteria.