University of Salford Academic Audit Methodology

ACADEMIC AUDIT METHODOLOGY 2013-2014

1. Introduction

The University of Salford's arrangements for academic governance and management were
radically reformed and changed from the start of the 2008/09 academic session enabling the
clear distinction of the respective roles and responsibilities of governance and management.

A key part of the reform of academic governance was the establishment of Academic Audit and
Governance Committee (AAGC) to be responsible for the effective auditing, monitoring,
reviewing and oversight of governance processes. AAGC combines in a single standing
committee of Senate responsibility for Ethics, Nominations and Academic Audit and oversees
the academic audit of the operation of regulations, policies and procedures and advises Senate
on the operation and effectiveness of academic governance and academic activities.

2. AAGC Terms of Reference

The Terms of Reference of AAGC comprise:

i. To oversee and report to Senate on the outcome of academic audits and the
implementation of any recommendations.

ii. To advise Senate on the operation and effective discharge of the University scheme for
academic governance and the appropriate delegation of Senates authority.

iii. To advise Senate on its terms of reference, composition, membership and standing
orders.

iv. To advise Senate on the establishment, terms of reference, composition, membership

and disestablishment of its standing committees and their standing sub-committees.

v. To monitor and report to Senate on the effective operation of the Senates standing
committees and sub-committees and time or task-limited working groups.

vi. To advise Senate on the development and periodic review of policy governing the audit of
its academic regulations, policies and procedures.

vii. To advise Senate on the development and periodic review of policy governing the ethical
approval of the academic work of the Universitys staff and students.

viii. To oversee and advise Senate on College and School schemes of management.

ix. To monitor the outcomes of and responses to internal reviews of the Universitys
academic units and to refer relevant issues to Senate and its standing committees.

x. To recommend to Senate the establishment, terms of reference and composition of any

time or task-limited working groups deemed necessary to develop, advance or review the
Committees business.

3. AAGC Members

The Governance website carries up to date lists of members of Senate and its standing
committees including AAGC.

4. Academic Auditors

AAGC recruits a pool of academic staff through nomination by Schools to conduct academic
audits. AAGC also invites the Students Union to nominate academic auditors. Recruitment is
against the following Person Specification:

Page 1 of 14
University of Salford Academic Audit Methodology

Role:

To assist in the quality assurance of the Universitys academic processes and, through this,
of the quality of the student experience.

Key Responsibilities:

reading and analysing University academic policies, procedures and other

documentation;

conducting structured interviews with relevant staff involved in managing or

administering University academic procedures;

with a team of peers, drawing evidence-based conclusions and making

recommendations and judgements on University academic policies and procedures;

drafting sections of audit reports for consideration by AAGC.

Knowledge and Understanding Required:

current experience of University academic policies and procedures such as:

- Academic Regulations
- Admissions
- Assessment
- Quality Assurance

knowledge and familiarity with the QAA Academic Infrastructure and other external
reference points such as Professional, Statutory and Regulatory Bodies;

familiarity with academic processes for improvement and enhancement;

service elsewhere within Higher Education for example as an External Examiner or on

secondment to QAA.

Skills Required may include the ability to:

conduct meetings and interviews with staff

conduct meetings with current and former students
write succinctly and coherently
meet tight timescales and deadlines
interpret statistics
work effectively as a member of a team
work courteously and professionally
maintain confidentiality
communicate electronically

AAGC invites nominated Academic Auditors to advise on their particular areas of interest and
expertise so as to inform AAGC's selection of audit teams. All Academic Auditors are offered
training.

5. Selection of Audit Topics

AAGC employs a number of mechanisms in selecting audit topics:

a call to the three other Standing Committees of Senate:

- Academic Programmes and Partnerships Committee

- Learning, Teaching and Enhancement Committee
- Research Committee

Page 2 of 14
University of Salford Academic Audit Methodology

requesting possible audit topics supported by a clear rationale for the nomination of the
particular topic;

liaison with Council Audit Committee and with the University's Internal Auditors, including at
least one meeting a year, to ensure that their respective audit programmes dovetail rather
than duplicate;

consultation with the Executive on which audit topics might best complement the
University's strategic priorities. The suggestion of topics for academic audit should be
included on the Executive agenda at least once a year. The Chairs of the three other cited
Senate Standing Committees are also members of Executive;

a balance between new topics and revisits to previously audited topics and between
existing long-standing procedures and new or prospective procedures;

preparation for forthcoming external review events such as formal visits by Professional,
Statutory and Regulatory Bodies (PSRBs) or the Quality Assurance Agency for Higher
Education (QAA);

committees may also propose as a topic a review of one of their own past resolutions as a
mechanism for assessing the effectiveness of committees;

allowance for contingency an urgent and unforeseen need to audit a particular topic.

A list of audits conducted by the former Academic Audit Committee and by AAGC is attached as
Appendix A for information.

Proposers of academic audit topics are requested to identify

which academic objectives within any appropriate University Strategy (such as the Learning
and Teaching Strategy or the Research Strategy) the nominated topic aligns to
how far reaching the topic might be, so what proportion of the overall student population
might be impacted upon.
whether any specialist skills or additional resources might be required to audit the
nominated topic

6. The Academic Audit Process

Appointment of an Audit Team

AAGC appoints an audit team for each selected audit topic from the pool of trained Academic
Auditors, taking account of individual areas of expertise and interest. AAGC also seeks to
share out the workload involved in assisting on audits so Academic Auditors will probably serve
only once, at most twice, in any academic session. AAGC nominates one member of the team
as Team Leader and the Head of Governance Services Unit (GSU) appoints a GSU officer to
support the audit as Review Secretary. On occasion, so as to broaden the experience of other
Professional Services staff and in discussion with their line managers, the Head of GSU may
invite an officer of a College, School or another Professional Service to support an audit as
Review Secretary. Training will be provided as needed.

AAGC also appoints one of its own members to each audit team. AAGC members who are also
members of the Executive are not eligible for appointment. The AAGC member will primarily
act as a facilitator, an adviser and a bridge between AAGC and the audit team. The AAGC
member does not actively engage in the detailed activity of the audit.

On occasion AAGC may have available for particular audits additional support for audit teams
through the use of external consultants. This is allocated as a set number of consultancy days
for a particular audit. Executive determines the level of external consultancy resource available
in any academic session.

Page 3 of 14
University of Salford Academic Audit Methodology

The Audit Proposal

The first task of an audit team is to agree an Audit Proposal comprising the particular Terms of
Reference and Methodology for the audit. This can be agreed by electronic exchange between
members of the audit team, facilitated by the Review Secretary and including the AAGC
member. A sample Audit Proposal is attached as Appendix B.

The Audit Proposal is approved by the Chair of AAGC who may consult the AAGC member of
the audit team and other members of AAGC.

Communication

Appropriate notification is composed by the Team Leader, supported by the Review Secretary,
and is sent to Schools/Units where an audit will take place outlining:

the remit of AAGC

the topic to be audited
the membership of the audit team
the terms of reference and methodology for the audit

The Schools/Units where the audit will take place are:

invited to offer any initial factual commentary they may wish on the audit;

requested to provide information on the location of any necessary documentation and key
contact persons;

invited to provide information on any relevant work already in progress or planned in the
area of the audit topic.

A sample notification is attached as Appendix C.

The Audit

The audit may include structured interviews, questionnaires, attendance at already scheduled
meetings, document review and agenda/minute trails. The detailed programme of activity will
reflect the methodology set out by the audit team in the Audit Proposal.

The Review Secretary assists audit team members by advising on appropriate contact persons
in the Schools/Units where the audit is to take place. However, audit team members will need
to arrange the timings of any visits to those Schools/Units according to their individual
availability.

The Team Leader, assisted by the Review Secretary, agrees with members of the audit team
which team members will cover particular aspects or venues. For avoidance of doubt, the
Review Secretary will circulate a note of the allocation of the respective tasks to all team
members.

The Team Leader, again assisted by the Review Secretary, agrees with members of the audit
team which team members will draft which parts of the audit report. Again the Review
Secretary will circulate that allocation in a note to all team members.

7. The Audit Report Process

a) Upon completion of the audit, the audit team agrees a draft report.

b) The Team Leader, assisted by the Review Secretary edits the draft audit report.
He/she may also consult the AAGC member of the team on editing matters. At this
stage of the process the Team Leader is likely to engage the rest of the audit team in
an interactive exchange of electronic drafts to arrive at a draft which reflects the
consensus view of all the audit team.

Page 4 of 14
University of Salford Academic Audit Methodology

c) This draft is then sent to the Head of School/Unit where the audit took place, and other
staff as appropriate, primarily for factual correction. Only where it is clearly
demonstrated that an error of fact has incorrectly influenced the audit team's findings
will the audit team redraft a report to the extent that its findings are amended. Again it
is for the Team Leader, with assistance and advice, further to edit the report.

d) The audit teams report, as amended if necessary in response to factual

correction, is then considered by the Chair of AAGC who may consult with the AAGC
member of the audit team, other members of AAGC and Governance Services Unit,
before determining:

which member of Executive should have overview of a draft management response

and should present that response to Executive ;

with advice from that Executive member, which University officer(s) should draft a
management response, under the overview of and for the initial approval of that
Executive member, for presentation by that Executive member to Executive .

e) As the key executive managers in the University, Executive have the authority to
commit staffing and other resources to ensure that recommendations in an audit report
are implemented. Executive members also have a commitment to and expertise in the
assurance of academic standards and quality and their enhancement. Executive
therefore consider the audit report and a draft management response, as presented by
the Executive member so requested by the Chair of AAGC, for approval.

f) After receipt of the management response, AAGC approves an action plan, drafted by
officers within GSU setting out the actions required to implement recommendations
within an audit report. The action plan identifies the locus of responsibility for specific
actions and sets out a timescale for those actions. The timescale will depend upon the
particular topic audited but AAGC adopts a default position of expected completion of
actions within twelve months.

A sample action plan template is attached as Appendix D.

AAGC then monitors its action plan until all actions are reported as completed. AAGC
receives an update on all current action plans at its final meeting of the academic
session.

A process chart is attached as Appendix E.

8. The Format of the Audit Report

Audit reports have a standard format comprising:

> Title of the audit.

> Membership of the audit team.
> Terms of Reference for the particular audit.
> The methodology of the particular audit.
> A description of the current policies, procedures and arrangements for the topic audited.
This should preferably be a brief factual synopsis, eg:

purpose of the topic audited

policies, procedures and arrangements in operation
background documentation or records

> A risk assessment outlining why the policies, procedures and arrangements
underpinning the topic audited require to be quality assured:

what detriment might the University suffer if the current policies, procedures and
arrangements failed?

Page 5 of 14
University of Salford Academic Audit Methodology

> The findings of the audit. These are principally of two types:

is there compliance with the current policies, procedures and arrangements for the
topic audited?

could the current policies, procedures and arrangements for the topic audited be
enhanced and be made more fit for purpose? In particular, the audit team may
wish to promote across the University good practice identified in the Schools/Units
where the audit took place.

> Confidence Judgements

Drawing on QAA practice for Institutional Audit up to academic session 2010/11, a

confidence judgement is accorded by the audit team to indicate the robustness of the
quality assurance policies, procedures and arrangement for the topic audited.

One of the following judgements is given in each audit report:

Broad Confidence

A judgement of broad confidence usually indicates that the area under audit:

utilises rigorous procedures to ensure quality and to maintain standards;

has procedures which are used effectively and consistently;
is able to identify and address any situation which has the potential to threaten
standards or quality;
that where appropriate, information produced is complete, accurate and reliable.

A judgement of broad confidence may be accompanied by a small number of

recommendations that are considered either advisable or desirable but none that are
considered to be essential.

Limited Confidence

A judgement of limited confidence usually indicates that there are some deficiencies in
the area under audit. This may be as a result of:

a notable weakness in the management or implementation of procedures;

problems arising which have not been duly noted or fully investigated;
prompt and appropriate action not being taken to rectify problems;

where appropriate, reservations exist regarding the completeness, accuracy and

reliability of the information made available to the public by the University.

A judgement of limited confidence is likely to lead to a small number of

recommendations that are considered essential, as well as a number that are
considerable advisable and desirable.

No Confidence

A judgement of no confidence usually indicates that there is substantial evidence of

serious and fundamental weaknesses in the area under audit and that this may impact
on the quality of programmes and the standard of awards offered by the University. A
judgement of no confidence will be reached if:

there is a shortage of or flaws in procedures used in the area under audit;

procedures are managed ineffectively;

where appropriate, information made available to the public by the University

cannot be relied upon and can be shown to be inaccurate and/or misleading.

Page 6 of 14
University of Salford Academic Audit Methodology

A judgement of no confidence will be accompanied by a significant number of

recommendations that are considered essential as well as a number that are
considered advisable and desirable.

> Recommendations

Again drawing on previous QAA practice, an audit report ends with recommendations
for action, categorised as follows:

Essential recommendations refer to important matters that the audit team believes
are currently putting quality and/or standards at risk and which require urgent
corrective action.

Advisable recommendations refer to matters that the audit team believes have the
potential to put quality and/or standards at risk and require preventive, or less
urgent, corrective action.

Desirable recommendations refer to matters that the audit team believes have the
potential to enhance quality and/or further secure standards.

Recent past audit reports are available on request electronically to assist audit teams in
understanding the standard format of an audit report. Please refer to Appendix A for details.

9. The Role of the Governance Services Unit and/or the Review Secretary

Officers of the Governance Services Unit support the academic audit process through:

the secretaryship of AAGC;

the maintenance of up-to-date information on the pool of Academic Auditors and their areas
of expertise and interest;

supporting the training of Academic Auditors;

A Review Secretary (whether or not a GSU officer) assists an individual academic auditor by:

advising on the format of an Audit Proposal;

notifying Schools/Units where the audit is to take place;

advising on appropriate contact persons;

recording and disseminating the division of responsibilities agreed between the audit team
members;

general progress chasing and facilitation during the audit;

advising on the format of an Audit Report;

assisting the Team Leader with his/her editing of the draft audit report;

Officers of the Governance Services Unit again support the academic audit process through:

upon receipt of an audit report and Executive Committee management response, drafting
an action plan for AAGC's approval;

progress chasing and updating approved action plans for monitoring by AAGC.

Page 7 of 14
University of Salford Academic Audit Methodology

10. Review Date

At the end of the 2015/16 academic session.

12. Further Information

For any further advice on Academic Audit, contact Gregory Clark, Secretary to AAGC, ext
55868 and g.clark@salford.ac.uk

Page 8 of 14
University of Salford Academic Audit Methodology

APPENDIX A

Titles of Audit Reports Produced by AAGC since 2009/10

1. Aspects of External Examining October 2009

2. Process of Ratifying Module Marks November 2010
3. Process Withdrawal of Programmes/Strategic Management of the Programme Portfolio within
Schools and Faculties November 2010
4. Personal Tutoring System: Personal Development Planning November 2012
5. Support for Students with Disabilities during Assessment and especially Examination
November 2012
6. Specialist Career Advice and Employability November 2012
7. Support for Placement/Internship November 2012/8. Consistency of the Universitys
Marking Criteria and Descriptors November 2012/9. Anonymous Marking audit
commissioned
10. PhD by Submission of Published Works May 2013
11. Processing of External Examiner Reports audit commissioned

An electronic copy of any Academic Audit Report can be obtained, on request, from Governance
Services Unit

From 2012/13 academic session Academic Audit Reports, Management Responses and AAGC
Action Plans were annotated for information to the academic audit team members for information

Page 9 of 14
University of Salford Academic Audit Methodology

Sample Audit Proposal APPENDIX B

Audit of the Conduct and Recording of Boards of Examiners

Rationale

Learning and Teaching Enhancement Committee (LTEC) requested this audit of Boards of
Examiners (BoX). The request arose from the consideration of academic appeals and the concern
that decisions made by BoX were not being recorded properly. There was also a concern that there
might be inconsistent practice in the ways that BoX were conducting their business.

Terms of Reference

The specific terms of reference for this audit are:

1. To determine practice at programme level through consideration of documentation provided by

Schools and to compare it to the standard University commended practice.

2. To determine if the minutes, spreadsheets and other relevant documentation are clear and
consistent and whether the reason for a decision is transparent and auditable.

3. To determine if the minutes record the correct procedure in relation to unfair means, debtors,
Personal Mitigating Circumstances (PMCs) and compensation.

4. To determine if Boards of Examiners are acting in a timely manner in the dissemination of their
decisions and their reconsideration, as necessary, at the next available meeting.

5. To identify aspects of good practice in the operation of Boards of Examiners and their recording
and make recommendations for enhancement.

Methodology

1. Assessments Office to collect a random sample of BoX minutes and associated paperwork from
across the University of postgraduate, undergraduate and foundation degrees to include two
programmes per school for a two year period.

2. The Audit Team to examine the minutes and other documentation provided by the Schools in
light of the terms of reference and seek further clarification as required.

3. The Audit Team to agree a draft report which includes results, recommendations for
enhancement and a confidence judgement.

4. The report to be circulated to Schools for a factual accuracy check and amendments made, if
appropriate.

5. The final draft report to be considered by Academic Audit and Governance Committee.

Page 10 of 14
University of Salford Academic Audit Methodology

APPENDIX C

Academic Audit and Governance Committee (AAGC)

MEMORANDUM

To Heads of School

From Officer supporting the audit

Date Insert Date

Audit of the Conduct and Recording of Boards of Examiners

Academic Audit and Governance Committee (AAGC) is a committee of Senate which assists Senate
in quality assuring the Universitys procedures. The Committee, which is chaired by Professor Martin
Bull, carries out audits not only to confirm compliance with University procedures, but also to work
with the University community on the enhancement and improved effectiveness of those procedures.
The Committee also oversees the implementation of any recommendations attached to the audits.

The Committee has decided to audit the conduct and recording of Boards of Examiners and I am
writing, as Secretary to the Committee, to advise that all Schools will be contacted for copies of
minutes and associated information for particular programmes. The Audit Team carrying out this
audit are:

Dr AB, School of the Built Environment (Team Leader)

Prof CD, School of Environment & Life Sciences
Dr EF (AAGC representative)
GH (Officer from GSU)

and it would be appreciated if you could inform appropriate staff in your School that they may be
contacted for information and if you would let me know the appropriate contact names and details.

If you wish to offer the audit team an initial brief factual commentary on the topic to be audited, that
would be most welcome. Any such comment or advice that you may have should be sent via me. It
would be particularly helpful to know in advance if there is work already in progress or planned in this
topic area.

The draft report produced by the Audit Team will be made available to you for factual correction
before being approved by AAGC.

If you have any queries about the audit or any information relating to it, either at this stage or at draft
report stage, then please do not hesitate to contact me.

Thank you in anticipation of your assistance to Academic Audit and Governance Committee in its
task of quality assuring and enhancing University procedures.

Page 11 of 14
University of Salford Academic Audit Methodology

Sample Action Plan APPENDIX D

Audit of the Conduct and Recording of Boards of Examiners

Recommendations for Enhancement Action to be taken by Completion Date Update

1. That the template agenda should be updated to include the highlighted good Assessments Office April. For implementation at
practice found in the programmes audited. That the template should be the Summer examination
accompanied by some guidance notes and circulated widely to all Board of boards.
Examiners committee Secretaries and Chairs. The master template should
be readily available electronically for ease of use.

2. That there is a clear need for staff development, not only in relation to chairing Heads of School December. Must identify
Boards of Examiners, but also in terms of committee servicing specifically chairs and secretaries of all
related to Boards of Examiners. It is recommended that a staff development examination boards and
session on minuting Boards of Examiners should be mandatory for all eligible alternative in case of
Secretaries. Schools should encourage all Board members to attend the sickness and ensure that
training session and regular updates as required. they receive mandatory or
refresher training.

3. A system of monitoring the minutes and spreadsheets of Board of Examiners College Deans June. For implementation
needs to be put in placed. It is recommended that the Colleges should carry after the Summer
out sample internal audits for their Schools. The audits should ensure that examination boards.
students can be tracked. These should take place after each session of
progression and awards Boards and the results reported to the College
Board. Colleges could make use of the methodology adopted for the
compilation of this Academic Audit and Governance Committee report.

4. There should be an investigation as to whether SIS: Banner or Gradebook Director of Student April. Need to investigate
could be developed so as to assist in the formatting of minutes, ensuring the Information and the feasibility and to advise
inclusion and correct identification of all students on the programme. This Assessments Office Executive on the resources
may need investment and needs to be completed before the next round of needed for implementation
Boards of Examiners. for summer examination
boards.

Page 12 of 14
University of Salford Academic Audit Methodology

5. Guidelines for a module board should be drawn up and made available Assessments Office April. For implementation at
across the university, which should include prompts for the minutes. the Summer examination
boards.

Page 13 of 14
University of Salford Academic Audit Methodology

APPENDIX E

Academic Audit & Governance Committee

Process for the undertaking of an Academic Audit

Request for potential

Liaison with Council Proposals for potential
audit topics circulated to
Audit Committee & audit topics received
other Senate
Universitys internal from the Executive and
committees by
auditors other sources
Secretary to AAGC

Details of
Suggestions for
forthcoming external
potential audit List of previously
review events e.g.
received by the audited topics
PSRB visits or QAA
AAGC
audits

No

Audit Team agrees

AAGC selects topic Audit Proposal, Audit proposal Team Leader and
and appoints Audit including terms of approved by Chair of Yes Review Secretary
Team reference and AAGC inform relevant units
methodology

Units provide
factual commentary
Team Leader
Information on key Audit Team Units and other staff
assisted by Review
persons and location undertakes audit as as appropriate invited
Secretary drafts
of documentation agreed in to offer factual
report for approval by
Information on work methodology corrections
Team.
in progress/planned
in area of the audit

If necessary Team Chair of AAGC

requests Chair of AAGC
Leader & Review
Executive member requests University
Secretary make
to take overview officers to draft
factual corrections
of and present management
and submit to Chair
draft management response
of AAGC
response

AAGC receive report

and management
Executive approve
response and officers AAGC monitors the
a management
in GSU draft an action plan
response
action plan for AAGC
approval

Page 14 of 14