The external assessor or evaluator has a key role and her/his evaluation will be an important factor in the rating. After the educational institution carries out self-evaluation, which is supplemented with the evaluation by its customers, it will be important to confirm that the entire evaluation process was carried out correctly and has the necessary validity. External evaluators fulfil this role. They should guarantee that individual evaluations will be comparable and that differences will not arise.
The key factors in the effectiveness and quality of the entire evaluation process will be not only the requirements for external evaluators but also clearly and comprehensively set evaluation processes and criteria.
The external evaluator will conduct the evaluation through:
- Preparation
- On-site visits and Interviews
- Customer evaluation
- Report with recommendations
Specific requirements for external evaluators include:
1) Basic assumptions:
- is fully competent to perform legal acts,
- has a clean criminal record
- his/her moral integrity or impartiality are not questioned
2) Requirements for professional competence
- university graduate – with at least bachelor’s degree in a preferred field of education, such as andragogy, management,
- proves 3 years of experience in a managerial position in the field of education and 3 years of teaching experience, or
3) Training for external evaluators
- has previously completed training for external evaluators in the area of education
For a successful evaluation process, it is important that the evaluated institution fully accepts the evaluator. This will be ensured by the rating institution that administers the whole process of evaluation.
The training for external evaluators is organized by the Qualtrack rating institution (project partner). Evaluators must pass an introductory seminar on the importance and benefits of various quality assurance systems and the rating system used. They will discuss the process of evaluation, the impartiality of the evaluator, the code of ethics of the evaluator, and the rights and obligations of the evaluator incl. confidentiality. The training should also include an exercise focused on solving sample case studies. Upon completing such training, the assessor will be awarded a certificate.
The role of the external evaluator is to go through the questionnaire together with the quality team from the assessed institution and check if the answers can be verified in regular documentation kept by the institution. The findings of this work will be entered into a report with a final judgment saying “Answers confirmed” or “Answers not confirmed”. The report will be kept by the rating institution.
A very important part of the evaluation process is observations and recommendations made by the external evaluator on the improvements to be made by the institution.
The evaluator shall also state if the institution qualifies for a badge in any of the examined areas. The badge signals that apart from fulfilling all the basic quality criteria, the institution is exceptional in something, such as the use of digital technologies, and deserves special recognition for that – a badge.
The visit will be done by two or three evaluators. For the piloting purposes, one evaluator is enough. Evaluators are also guides and they help the institutions to understand and fulfill the criteria. To this end, they may agree on a pre-visit in the first phase of the evaluation process, and then the final evaluation visit can take place
The evaluation process
- Preparation – is based on studying the self-assessment questionnaire of the institution and checking all the support documents. It will give rise to the agenda of the evaluation visit.
- On-site visit – will be prepared in advance, with an agenda setting up a timetable and persons with whom the evaluator meets, mainly the members of the quality team at the institution. The evaluator will be interested in documents such as the Mission/Vision statement, Statutes, Internal Guidelines (including discrimination clause, GDPR rules, etc.), Communication Plan, Annual Report, Code of Ethics, Business Conditions, Complaint Code, Working Conditions and similar. During the visit, a set of interviews will be conducted with the representatives of various groups in the institution, who can supply the information needed, such as management, teachers/trainers, programme administrators, and students/clients. The questions will be directed towards the issues that were not clear in the answers to the questionnaire.
- Customer evaluation: The evaluation of the work of the education institutions by customers is an important indicator of the quality of its work. The first step is that the institution is interested in collecting such evaluation or feedback from customers, next thing is how it applies it to further improve its work.
The customers are twofold – the employers who order some educational programmes for their employees or direct beneficiaries of the education, the participants. It is assumed that an educational institution has a system in place that collects data on the client’s satisfaction. The evaluation process will review the existing system, there is no need to establish a specific system or collect specific data just for the sake of the evaluation process. The external evaluator will ask for and use the data from the feedback process that is regularly used by the educational institution.
- Final report and recommendations – during the evaluation process the evaluator gets to know the institution rather well. It is an added value added of the whole evaluation exercise for the institution, that the evaluator shares the observations and most importantly the recommendations. The institution can then make a plan for the future and improve its internal processes.
General rules for an evaluation visit
- Evaluators follow the agreed-upon schedule.
- Evaluators follow the rules of the interview.
- Evaluators treat entrusted data and feedback as confidential information.
- Evaluators try to listen with understanding and make objective conclusions.
- Evaluators work as a team and support each other.
- Evaluators are prepared for unforeseeable events.
Rules of communication and interview
- The evaluators’ attitude should be based on an exploratory, open, and accepting approach, striving to understand without hasty interpretations and judgments.
- Use appropriate language.
- Establish contact, take an open and friendly attitude, actively listen, and show interest.
- During the interview, speak briefly and clearly, ask short and straightforward questions, and do not interrupt the respondent’s thought process.
- Do not speak too much, so that the respondent has as much space as possible.
- Ask for facts and examples, especially if the answers are too general, ask for additional explanations.
- End the interview on time according to the plan or specific situation.