NPC/02/11/A: Joint Funding Bodies Review of Research Asessment

by Tim Brown

Joint Funding Bodies

Review of Research Assessment

Consultation paper

Response by the National Postgraduate Committee

On behalf of the National Postgraduate Committee (NPC) I am pleased to respond to the joint funding bodies consultation paper on research assessment. This was discussed at a meeting of the NPC held on 3rd November 2002 at the University of Cardiff. I am grateful for the supporting comments conveyed by the NPC for this consultation.

Tim Brown
General Secretary
National Postgraduate Committee

29/11/02

Executive summary

As will be found in answer to the questions laid out in the consultation paper, we support a peer assessment approach to reviewing research quality. Therefore we identify research quality in terms of the contribution that the research has to the wider international community it is contributing to. Likewise quality research should be widely published to such an audience and we would also support this going beyond that to express its significance to society in general. As students we would also identify high research quality in terms of the management and development of researchers in a unit.

Preamble

The National Postgraduate Committee (NPC) is a charity with the aim of furthering UK postgraduate education in the public interest. We organise meetings and conferences, publish best practice guidelines and seek to influence public policy on all aspects of postgraduate education. Our membership consists of affiliated students unions from across the UK; we have one full-time officer, the General Secretary, and fourteen voluntary officers. We work closely with the National Union of Students and the lecturers unions as well as other bodies relevant to postgraduate education.

Responses to the Suggested Assessment Methods

With regard to the four different methods of research assessment put forward (i.e. expert review, algorithm approach, self assessment and historical ratings) we have the following responses:

Expert Review

We favour the need for an expert review since the value of research to people in the wider community can only be identified from a peer assessment. Research varies considerably from subject to subject and so we would consider that assessment should vary likewise. The assessments, we believe should be a combination of a prospective and retrospective review. In the case of older research units there may be more emphasis on retrospective assessment, where as for a new and emerging unit it may have more prospective assessment; therefore flexibility is required.

The assessment of a research unit should, we feel, primarily consider the usefulness of the contribution to knowledge generated so that it can take a lead in world class research. Further to this we consider the quality of a research environment in terms of the management and development of the researchers in the unit. We therefore agree with paragraph 10, which mentions this specifically. Finally we would support the research being communicated to people beyond the research community so that it creates awareness to society in general of the significance the research has. This may be of benefit to society in a number of ways such as increasing public interest in education and widening participation.

With regard to the level at which a research assessment should be made, we support that it should be carried out at subject level as is currently the case because we support a peer review assessment. Further to this we would support the fact that research should be assessed by an appropriate board suitable to the specialist research areas in a unit. We do not support the idea that there could be an alternative to subject level assessment. The quality of research cannot be identified without considering its place in the specific research community. Further to this, an assessment in a broad subject (e.g. Engineering or Biological Sciences) would prove inappropriate since it would constitute many units of varying quality. A specific subject (e.g. Electrical Engineering and Toxicology) would be desirable to determine which specialist subjects an institution is strong in.

This is a detailed enough investigation to tell the wider community with which research units have the most appropriate resources and specialist expertise in a given subject. The disadvantages are, however, the time and the detail at which research units need to be evaluated. In depth subject level research does also cause some difficulties in terms of whether all subjects have consistent assessment. Without such detail into a particular area of research, however, a more simplified approach may cause the assessment to be unfairly justified so that research with high potential could be prevented.

Algorithm Approach

We appreciate that an algorithm approach may well allow some degree of ease to a research assessment. Further to this it would allow there to be some consistency in the assessment. However, this approach we feel would not perform consistently across all subject areas. Each subject area would need to be assessed individually in different contexts. Because the separate areas are restricted in some way or other, we would consider that this approach would not be thorough enough for an entire assessment although may be useful to some degree. Reliance on this method, we feel could prevent some positive aspects of research to be unidentified and resultantly not well funded.

Assessment of research could be carried out in terms of whether previous funding has been well invested, management structures to develop and equip researchers, whether targets have been successfully met and the support from industry and other outside bodies.

Self Assessment

A self assessment, if implemented would be produced by a written report as far as we can identify. This report would indicate clearly identified targets and clear plans for the future along with details of collaboration with work outside. The detail of previous work and future plans would be dependant on the age of the research unit. Following this we would expect an audit of some form to take place in order to moderate the assessment.

We feel that this approach would have difficulties for many institutions for two main reasons. The first, we consider would be that it would be difficult for a research unit to identify their weaknesses and many others will be reluctant to show them. In addition to this the burden of time and effort required to produce a self assessment would be overwhelming on top of the pressures in academia. However, if reports were produced by research units for other purposes, we would encourage that these are used within an assessment for useful background information.

Historical Ratings

This approach, we feel, would be inappropriate as a measure of research, especially for new and emerging units. It is likely that this could lead to a serious reduction and possibly dissolving of smaller research units, which may fail to improve from lack of funding. For those with a good historical record, it may be the case at present that they remain at the same level of standard although such assessment could encourage complacency and slowing down of research.

Responses to the Crosscutting Themes

With regard to the crosscutting themes, our answers to the questions asked are as follows:

a. What should/could an assessment of the research base be used for?

As well as a funding base we consider it as a way for outside bodies to identify a comparison between different units and if the ratings are influenced by location. Further to this it gives some indication to students as to the potential of a given teaching or research programme at an institution.

b. How often should research be assessed? Should it be on a rolling basis?

We would support the fact that the frequency of assessment should remain every 5 years. This would provide sufficient time to plan and implement appropriate targets or improvements necessary for successive assessments.

c. What is excellence in research?

We consider this to be significant world class contribution and well published or disseminated information. This would provide suitable contribution to the wider research community. We would also support publicity to widen information to other groups not in the research community.

d. Should research assessment determine the proportion of the available funding directed towards each subject?

We would support a more streamlined allocation to emerging subjects. This would therefore prevent the problem of such units being unfairly overpowered by larger research units. Therefore this provides an increased opportunity for such units to grow and offer the research that may otherwise be not offered elsewhere.

e. Should each institution be assessed in the same way?

We would consider that assessment should not be necessarily the same for those who are smaller or emerging. There would be a wider assessment of the future work in this situation in terms of where development needs to be supported. In the case of older and established research units, they would be required to be accountable as to whether they are productively using their funding or rather absorbing it unnecessarily.

f. Should each subject or group of cognate subjects be assessed in the same way?

We would not support this since we are in support of a more appropriate peer review on a particular subject, which will be researched with a particular approach. Therefore we would seek peers to assess the subject appropriately.

g. How much discretion should institutions have in putting together their submissions?

We have no comment here since this does not affect us.

h. How can a research assessment process be designed to support equality of treatment for all groups of staff in Higher Education?

Here we would support an appropriate procedure to moderate the assessment of research active staff. This would allow any inconsistencies with the grading of particular research staff to be identified, particularly from those working closely. We would also possibly support a complaints procedure if such difficulties were identified.

i. Priorities: what are the most important features of an assessment process?

We have no comment here.