This is a two-part essay on the role of student evaluation on teaching. The previous part addresses issues related to its educational merit. This week, I will address the role of the faculty in establishing assessment and evaluation policies.
Last week I addressed the plan of Drexel University’s Office of Institutional Research, Assessment, and Effectiveness, to institute an online, centrally administered, across-the-board, uniform, one-size-fits-all, centralized depository system of student evaluation of teaching. Here I will explain the role of administrators in teaching evaluation and why faculty should be the driving force for formulating a comprehensive and holistic system of assessment and evaluation of our educational programs.
In most colleges and schools at Drexel, professor’s teaching performance is measured largely by student evaluation of teaching, and these SET data are normally reviewed by the academic administrator of the program.
Yet, in my 37 years of teaching at Drexel, I was never asked to share any of my course content, classroom practices, course syllabi, examinations, lecture notes, handouts and other instructional aids, except for filing the required material prior to visits by our accreditation association. My colleagues share the same experience. None of my 11 department heads ever requested to attend my lectures. Most information about my teaching performance is based on students’ word of mouth or the SETs, which have an average return rate below 25 percent.
The academic administrator (department head, dean or provost) evaluates teaching performance, often based solely on SET. It is highly presumptuous for any AA to evaluate the teaching effectiveness of a faculty member based primarily upon student opinion.
First, let’s distinguish between AAs evaluating faculty scholarship and evaluating faculty teaching. In my own college, engineering, faculty scholarship is based essentially on quantitatively counting numbers of publications, citations, impact factors, numbers and degrees of students supervised, number and size of external grants, etc. These are all numbers that the department office staff can easily collate.
The AA infers the quality of the scholarship by relying on the strict scrutiny of the professional peer review process. That is, the AA relies on other professionals, mostly external, to evaluate the scholarship of a faculty member. This process is common and satisfactory; very often the AA is not familiar with the detailed disciplinary area of the scholars in their academic unit, particularly so in multidisciplinary departments.
The evaluation of instruction, however, is a different beast altogether. This can only be done qualitatively. This is what we mean by evaluating “quality of instruction.” Quantitative evaluation of teaching is primarily related to workload distribution.
So, what is the role of the AA in teaching evaluation? It is in my experience and that of a large number of my colleagues across campus that the AA relies almost entirely on the course SET, largely ignoring other well-established teaching evaluation methodologies. The job of department heads (the front line AAs) is the toughest on campus: they are simply inundated with priorities, such as day-to-day running the department, ensuring greater scholarly productivity, supporting their faculty and attending to regularly occurring crises.
Establishing the complex professional methodologies to evaluate the quality of instruction is not the AA’s top priority. Besides, few if any AAs are trained in the complex area of assessment and evaluation (a professional discipline by itself).
We confront, therefore, the Great Academic Predicament of instructional effectiveness: the AA, who normally is not a trained educational evaluator and is often unfamiliar with the detailed course content, is obliged to review a huge pile of student course evaluations submitted by inexperienced young students who provide — in the aggregate — often erratic, random and statistically meaningless data about the instructor and course.
Further, the AA is expected to review voluminous course evaluations prior to the faculty annual review. I estimate that in my department (of 29 faculty, 26 of whom are tenure-track or tenured), the head is expected to review some 2,000 SETs (considering 25 percent return rate) evaluating some 150 courses, where each SET includes 14 questions and four comments, yielding a total of 28,000 numerical data and 8,000 comments.
I wonder how many SET forms the AAs actually review, and to what depth. Unequipped “customers” submit this mountain of data in the last few days of each term. Most tenured faculty members do not take them seriously, and they are rarely fully reviewed by the AAs. Therefore, this enormous operation yields a limited return of questionable value.
By all established professional principles, the faculty should have the primary responsibility for devising a system of assessment and evaluation of teaching effectiveness. There are three fundamental tenets:
- The Charter: The authority and responsibility for constructing and delivering a curriculum rests solely with the faculty, as is articulated in the board-approved Charter of Faculty Governance. Article 3.1: “The University Faculty is authorized by the Board of Trustees to formulate rules and regulations directly governing such matters as the … curriculum; admission standards; scholastic standards; examination and testing programs …”
Further, “All members of the faculty are guaranteed full academic freedom in the discharge of their University functions, as defined in the 1940 Statement of Principles of Academic Freedom and Tenure formulated by the Association of American Colleges and the American Association of University Professors” (article 15.1), and “all faculty evaluations shall be carried out according to the norms, criteria, and standards expressed therein, and that their application shall be substantive at each stage of the evaluation process” (article 15.2).
The Charter of Faculty Governance has been duly approved many times over the past two and a half decades by Drexel faculty and the Board of Trustees. The University fully adheres to its curricular provisions. Similarly it has been Drexel’s long-standing policy, like all reputable universities in the United States, to adhere to the numerous provisions and guidelines of the AAUP vis-a-vis academic freedom and due process.
Clearly, assessment of instruction is a strictly curricular matter. Therefore, the formulation and implementation of all related policies are the sole prerogative of the faculty, as are the corresponding applications, analyses and outcomes assessments.
- The AAUP: The AAUP’s 1975 Statement on Teaching Evaluation (expanded in 2006), recognizes that “the most valid measure is probably the most difficult to obtain, that is, the assessment of a teacher’s effectiveness on the basis of the learning of his or her students.” It is difficult because “student’s learning is importantly influenced by much more than an individual teacher’s efforts” and “measures of before-and-after learning are difficult to find, control, or compare.”
Regarding the tenets for evaluating teaching performance, the AAUP expects data for evaluating teaching performance to be based on “trained observers, faculty colleagues and students.” While soliciting “student perceptions” (i.e., SETs) is “a prime source of information,” nevertheless “faculty members should be meaningfully involved in any systematic efforts to obtain student opinion. Cooperation among students, faculty and administration is necessary to secure teaching performance data that can be relied upon.”
The AAUP advocates that “the emphasis in evaluation should be upon obtaining first-hand evidence of teaching competence, which is most likely to be found among the faculty. … It is the responsibility of the institution … to establish and maintain written policies and procedures that ensure a sound basis for individual judgments fairly applied to all. … Faculty members should have a primary, though not exclusive, role in evaluating an individual faculty member’s performance as teacher.”
The current teaching evaluation system, using SET as the primary teaching evaluation tool and reviewed solely by an academic administrator, circumvents key provisions of the Charter, and it sidesteps nearly all the fundamental tenets advocated by the AAUP. We should reinstitute regular academic order and abide by the principles clearly stated in the Charter and by the AAUP guidelines on teaching evaluation.
- Drexel Assessment and Evaluation: Drexel’s faculty has always played a key role in all institutional and programmatic assessment and evaluation related to self-studies for accreditation. The University and many of its colleges, schools, departments and programs are subjected to periodic external program reviews, accreditation, by the corresponding accreditation associations.
These accreditation agencies are recognized by the U.S. Department of Education and the Commonwealth as the authorities for ensuring quality and germaneness of the institution’s educational programs. The expected outcome of each accreditation process is that the academic programs of the institution are certified to accomplish its expected standards, objectives, quality and probity.
This includes, for example, the self-studies in preparation for accreditation by associations such as the Middle States Association of Colleges and Schools at the university level and many at the college level: e.g., the Association to Advance Collegiate Schools of Business for the LeBow College of Business; the Accreditation Board for Engineering and Technology for the College of Engineering; the National Association of Schools of Arts and Design for the Antoinette Westphal College of Media Arts and Design; the American Bar Association for the Thomas R. Kline School of Law, etc.
Internally, three years ago Drexel instituted its comprehensive Program Alignment and Review and has long instituted several mechanisms to ensure sustainable instructional quality. The faculty recruiting process is subject to intense scrutiny of the teaching qualifications of the applicants. A more elaborate review mechanism is applied during the tenure and promotion considerations of the candidates. Undergraduate and graduate student committees play vital roles in both reviews.
Further, the instructional performance and commitment to education of every faculty member are reviewed annually, through the well-established faculty annual review. The review examines the instructor’s performance in teaching, scholarship and service activities, nearly equally weighted for tenure-track faculty, while for the teaching faculty the primary evaluation addresses teaching accomplishments and effectiveness.
The primary role of the faculty in all curricular matters is well-embedded in Drexel’s tradition and clearly codified in the Charter of Faculty Governance, following the AAUP’s guidelines. Further, a large number of faculty members routinely play the major role in all accreditation, evaluation and assessment activities at all levels of the university.
SET has an important, but small, role in the overall assessment and evaluation program at Drexel. Of note is that Interim Provost James Herbert has informed me that he is establishing a University Advisory Committee to address assessment and evaluation issues at Drexel.
We must assure that SET plays a constructive role in our community of scholars dedicated to the pursuit and transmission of knowledge and understanding. To this end, administrative response to poor SET does not offer any corrective measures, especially if the faculty member is tenured. Instead, our students will benefit if Drexel institutes a faculty renewal program, structured for individual faculty members.
Jonathan Awerbuch is a professor of mechanical engineering at Drexel University. He can be contacted at [email protected].