NPC/01/10/B: HEFCE/QAA/UUK/SCOP Consultation on Quality Assurance

HEFCE, QAA, Universities UK and SCOP

Quality assurance in higher education

Response by the National Postgraduate Committee

On behalf of the National Postgraduate Committee I am pleased to respond to the HEFCE, QAA, Universities UK and SCOP consultation on quality assurance in higher education. I hope you find our comments helpful.

James Groves
General Secretary
National Postgraduate Committee

30/10/01

Executive summary

We regret the decision to abandon the audit model developed in Quality assurance in UK higher education: a brief guide (QAA, 2001) which we feel was not given sufficient time to prove itself. Further development is needed on the proposed new audit model to ensure all stakeholders, including students, are able to properly raise their concerns with audit teams. We are sceptical whether accurate reporting is possible on either overall academic standards or the quality of learning opportunities without a comprehensive system of review at the subject level.

Preamble

The National Postgraduate Committee (NPC) is the representative body for postgraduate students in the UK. We organise meetings and conferences, publish best practice guidelines and seek to influence public policy on all aspects of postgraduate education. Our membership consists of affiliated students unions from across the UK; we have one full-time officer, the General Secretary, and fourteen voluntary officers. We work closely with the National Union of Students and the lecturers unions.

The NPC has always shown a keen interest in matters of quality assurance; we were strong supporters of the HEQC and remain strong supporters of the QAA.

Responses to specific issues

The objectives and principles of quality assurance (paragraphs 7-11)

We believe the objectives suggested in paragraph 7 are too vague; at no point is a detailed description of the terms quality and standards given. We recommend the adoption of the outline of academic review given in the QAAs 2001 publication Quality assurance in UK higher education: a brief guide. This stated:

"The academic review process addresses three interdependent areas:

  • reporting on academic standards is concerned with the appropriateness of the intended learning outcomes (in relation to relevant subject benchmark statements, the qualifications framework and the overall aims of the provision); effectiveness of curriculum design and assessment arrangements (in relation to the intended learning outcomes); and the actual achievement of students;
  • reporting on the quality of learning opportunities in a subject is concerned with the
    effectiveness of teaching, learning resources and academic support in promoting student learning and achievement;
  • reporting on institutional management of standards and quality is concerned with the robustness and security of processes and procedures relating to the institution's responsibility as a body able to grant degrees and other awards that have a national and international standing. This involves, in particular, arrangements for dealing with approval and review of programmes, the management of academic credit and qualification arrangements, and the management of assessment procedures."
    (QAA, 2001)

We disagree with the assertion in paragraph 8 that it is no longer necessary to maintain comprehensive external review at subject level. Of the areas just cited, it was stated by the QAA that: the first two areas are addressed by reviews at subject level, the last by reviews at the level of the whole institution (QAA, 2001). We agree, and believe it will not be possible to continue effective monitoring of either academic standards or the quality of learning opportunities without comprehensive review at subject level.

We also do not see how stakeholders can have information which is up-to-date, consistent and reliable (paragraph 9) without an ongoing and comprehensive system of review at subject level. Just because a subject centre demonstrated robust arrangements for quality assurance in the past does not mean it will continue to do so though we agree the intensity of future reviews should be reduced.

We believe that, in addition to principles 9a, 9b and 9c, a principle of responsiveness should be established. Any stakeholder especially a student should be able to raise their concerns with a QAA review team, in the expectation that these will be investigated.

We agree with the lightness of touch principle, but do not believe this rules out comprehensive subject assessment, as you suggest in paragraph 10b. It is right that the intensity of a review, both at subject and institutional level, should be based inversely on the robustness of approach demonstrated in previous reviews. This does not mean reviews should cease altogether!

The external audit model (paragraph 12)

It is suggested that the quality assurance model should be analogous to external financial audit of organisations with well-developed internal audit systems. We believe it is extremely rare for HEIs to have systems of internal quality assurance that are as developed or thorough as their systems of internal financial audit. Indeed such a system of internal quality assurance would be very unwieldy to manage since, unlike financial audit, quality assurance necessarily includes elements of peer review.

We therefore do not believe such a comparison is appropriate.

The collection and publication of information (paragraphs 13-16)

We agree that the base room approach (paragraph 14) is unnecessarily bureaucratic.

We believe the internal information set (paragraph 15) should include a formal submission from the students union; this would complement the student feedback information supplied. The advantage of a students union submission is that it would give the student body a chance to set out, in clear and concise terms, areas where it believed further investigation was necessary.

The approach to institution-level audit (paragraphs 17-28)

As we intimated in our response to paragraphs 7-11, we regret the decision to abandon the audit model developed in Quality assurance in UK higher education: a brief guide (QAA, 2001) which we feel was not given sufficient time to prove itself. We feel an explanation as to why this model has been abandoned should have been included with the consultation document; as it stands, we have had to rely on the media for information.

The remainder of our response will offer our views on the proposed new audit model, assuming this is adopted in some form.

It is clear that drilling down (paragraph 25) will be a crucial part of future audits. We are concerned, therefore, by the vagueness of paragraph 26 regarding which subjects will be selected for drilling down. It is stated: as part of the initial discussion between the QAA and each HEI about the timing and handling of its institution review, a small number of subject areas or themes would be identified for more detailed review. In our opinion:

  1. Other stakeholder groups within the HEI, not just the management, need to be part of the discussions surrounding which subject areas are selected. Student representatives in particular should have the power to trigger drilling down in subject areas they have concerns about.
  2. There need to be clearly laid procedures and criteria regarding the selection of subject areas. The consultation paper suggests no criteria other than: the QAA would aim to secure that a balance of provision was examined, across the whole sector, in each of the full range of 42 subject areas during a full five year review cycle. This is not sufficient. We suggest that some subject areas are chosen following discussions with stakeholder groups, while others are selected at random by the QAA.
  3. We do not believe it helpful to adopt as a planning assumption that it would be sufficient to select areas accounting for a maximum of 10 per cent of the HEIs students for drilling down. If the drilling down approach is to be chosen, the proportion of subject areas selected will depend very much on the HEI. Where significant concerns are expressed about a high proportion of an HEIs subject areas, it may well be necessary to select more than 10 per cent of subject areas for drilling down.

The objectives of, and approach to, selective reviews of subjects or themes (paragraphs 29-34)

Our concerns mirror those on the approach taken to drilling down. In paragraph 32 it is stated: the size and structure of the follow-up programme, if any, would be discussed between the HEI and the QAA, involving the members of the audit team. There is a need for transparent procedures and criteria that would choose subject areas for follow-up work. All stakeholder groups in the HEI, including students, should be able to raise their concerns with the audit team.

Sensitivity must be shown towards staff in subject areas selected for follow-up work; it would be all too easy for them to believe they were being victimised by the review team.

We have a more fundamental point to raise. Under the proposed scheme there will be two distinct means by which subjects will be selected for review drilling down during institutional audit visits and follow-up reviews after institutional audit visits. We believe this could lead to duplication of resources.

It must surely be possible to combine drilling down with follow-up activity in a single subject review scheme operating under a single set of terms of reference. Under this single scheme, some subjects would be identified for review early on in the institutional review process (and so would be subject to drilling down) while others would be identified for review immediately following the institutional review process (and so would be subject to follow-up reviews).

We concur with your comments on accreditation by professional and statutory bodies (paragraphs 33-34).

Quality enhancement (paragraphs 35-38)

We agree that information collected by the QAA during audit visits should be shared with other bodies, particularly the LTSN, as a means of promoting good practice and enhancing quality.

The length of review cycle (paragraphs 39-40)

We broadly agree with the proposal in paragraph 39 that: the first cycle of institution audits should take place over a three year programme, and then move to a five year programme in steady state.

The approach to reporting and follow-up (paragraphs 41-45)

We agree with the proposed approach to reporting, but seek clarification on one point will all published reports, including conclusions of any follow-up reviews, be publicly available, as they are now? We feel this is crucial for the public accountability of HEIs.

Implementation and planning assumptions about the volume of activity (paragraphs 46-54)

We are sceptical about the claim that, given effective institution-wide review, and a purposive approach to follow-up reviews targeted on areas of concern, we believe a greater reduction sector-wide can now be achieved of at least 50 per cent (paragraph 51). As we mentioned in our response to paragraphs 17-28, we do not believe numerical planning targets are useful some HEIs may be able to achieve a reduction of at least 50 per cent, but others may not.

More generally, we do not see why a reduction in subject-level activity is necessarily to be celebrated. Certainly we oppose unnecessary bureaucracy. However we do not feel this consultation paper has demonstrated why comprehensive subject-level review should be abandoned; in the absence of such evidence, we believe it should remain.

Reference

QAA, 2001 Quality assurance in UK higher education: a brief guide, Quality Assurance Agency for Higher Education, 2001
www.qaa.ac.uk/public/heguide/review.htm