Periodic evaluations of study center sites and programs are essential for the ACB to maintain high-quality standards, particularly for academics. ACB evaluation reports are available to the Consortium to provide information to prospective students and sending institutions.
ACB members review the following points to determine whether each program serves the needs of CIEE’s primary stakeholders, Consortium-member institutions, and their students:
- What needs are served by the program?
- What is the site and program's role in CIEE's overall mission
- Is the program academically sound?
- Is it viable?
- Do activities support its goals?
- Are local resources well used?
- Are the programs academically sound?
- Do the programs have appropriate support, from both local staff and CIEE headquarters?
- What internal processes are in place to evaluate the quality of academic and support services?
CIEE publishes all program evaluations on its website for easy reference.
General principles of evaluation
- ACB members review each program according to its mission statement.
- Reviews must be regular and systematic. Each program is internally reviewed annually by CIEE staff and monitored by the ACB. Comprehensive on-site evaluations of each study center are conducted at least every 10 years.
- Reviews must include internal and external components. We believe self-evaluation is the best guarantee of program quality and improvement. For this reason, ACB study center reviews are conducted with full participation of the staff and students, but members of review panels are not associated with the programs they review.
- The review process must not impose an undue burden on staff nor impede the normal pattern of teaching and assessment.
- Because evaluation provides opportunity for improvement, final evaluation reports should be useful for planning.
Nature and scope of review
All reviews are conducted in accordance with the strategic objectives of the CIEE Academic Consortium Board and the stated purpose of each program. Each review reports on the following terms of reference:
- Quality of the academic activities, including teaching, assessment, and the appropriateness of the offerings
- Quality and appropriateness of services (predeparture and on-site orientation, academic and personal/intercultural counseling, housing, field trips, internships, accommodations for students with special needs, medical, etc.)
- Quality and appropriateness of the students who participate
- Quality of the site’s facilities (office and class space, library resources, equipment)
- Plans for improving the center’s teaching and services
- Relationship of center with the host university
- Perceived satisfaction of the Consortium-member institutions
- Perceived satisfaction of the student participants
- Safety and security
- Compliance of program and center with CIEE policies
- Quality of administration and management
- Future prospects and plans
- Relationship of this evaluation to previous evaluations
Before each review, the ACB evaluations coordinator, ACB evaluation team leader, team members, members of the CIEE Study Abroad executive team, and CIEE regional director of operations for the specific site being evaluated confer on whether more terms of reference are necessary, and if strategic issues or particular problem areas should be noted.
The size of each ACB evaluation team is determined by the size of the study center and the number of programs it offers. Each team has a leader who is a member of the ACB. Team leaders are appointed by the ACB chair and the ACB evaluations coordinator. The team leader appoints other team members from the faculty of Consortium-member institutions, and is responsible for conducting the review and overseeing the final report.
Individuals in the following positions are integral to the review process, but cannot be members of a review team:
- CIEE executive director of program management and operations
- CIEE executive director, program development and evaluation
- CIEE vice president for academic affairs
- CIEE regional director of operation for the site
- Other CIEE staff
- CIEE resident director of the study center under review
- current student participants
- faculty and administration of the host institution
The review process
ACB evaluation reviews include the steps outlined below. The process may take as long as six months.
Review information. CIEE provides each member of the review team with thorough information about the programs offered at the site.
Gather feedback from sponsoring institutions. After reviewing materials, the team conducts an electronic canvas of the top sending institutions, the school of record, and a sample of other sending institutions. If possible, the team leader conducts interviews with faculty and staff at some institutions by telephone, particularly if email communication has not been productive. Team members review this feedback to determine the institution’s level of satisfaction with the program, identify concerns, and assess how well the institution’s needs are being met.
Request feedback from students and program staff. Before their visit, team members invite on-site students and staff to meet during the visit and submit written reviews of the programs offered at the site to the panel.
Conduct site visit. This is the most important step because it gives the review team direct contact with participants, staff, and professors. A typical site visit takes five days. The team arrives on Sunday in time for a welcome meeting and dinner, followed by three days of evaluation, a day to prepare the team’s report, and departure on Friday. The visit includes intensive interviews, discussions, and deliberation, followed by formulating conclusions and writing a report. Team members must be available exclusively for the review during the site visit, as well as after the visit for consultation while the report is drafted.
Prepare team report. Team members are busy professionals with full workdays, so it’s essential that each report for the ACB be drafted on site before the team departs. The team leader oversees production of the report, which addresses each of the terms of reference, makes recommendations for change or commendations as appropriate, and includes an executive summary. Reports are submitted to the ACB chair and its evaluations coordinator and to the CIEE executive director of program development and evaluation (who then circulates it to CIEE staff) within two weeks of the completion of the site visit. The report must be detailed, complete, and frank in every respect. Although parts of the report later will be made public, the complete report will be read only by CIEE staff and ACB members.
Prepare ACB evaluation report. The ACB reviews completed reports at its next scheduled meeting.
Each ACB evaluation report serves as a baseline for the next evaluation of the program, as well as for the yearly monitoring that occurs between evaluations. Over several years, a dossier is created, documenting how the program measures up to CIEE's high standards for quality, service, responsibility, and efficiency.
CIEE develops action plan. CIEE staff members create an action plan in response to recommendations in the ACB evaluation report. The plan lists steps to address the major recommendations of the evaluation’s executive summary.
Regional directors follow up. To assure maximum benefit, CIEE regional directors of operations (RDOs) submit annual critical component reviews, commenting on the progress of the improvements.