BACKGROUND AND HISTORY OF SET

As early as 1984, UP Diliman had considered designing an evaluation-of-teaching instrument toward improving the quality of teaching in all units of the University. The initial version of SET was based on common aspects noted in a survey of evaluation instruments used in different UPD colleges.

The questionnaire has undergone several revisions and modifications since the first try-out version in 1987. In the early 1990's, the SET became one of the bases for identifying professorial chairs; while in some colleges it was used as one of the basis for decisions on faculty promotion and reappointment.

Since September 1992, SET has been regularly administered in the colleges.

Over the years, results of Part III (on Instructional Behavior of the Teacher) have been used in all lecture-oriented courses.

The SET continues to be studied. For one, its inadequacy as an evaluation instrument for laboratory, practicum and skills-oriented courses has been repeatedly pointed out.

There appears to be a need for a supplementary part which addresses aspects of teaching in skills, laboratory and performance courses.

 

 

THE FORMAT

The current SET form is divided into three main parts: 1) the student; 2) the course; and 3) the teacher.

Part I asks about the student's participation in the course, the extent to which his expectations have been met and his self-rated performance.

Part II elicits his perceptions and feelings about the course, its pacing, and the extent to which the course syllabus was followed. This part includes open-ended questions to enable the student to express his views freely.

Part III inquires into the teacher's method of teaching. It consists of 26 statements with which the student either agrees or disagrees mildly or strongly. Part III also elicits information on the teacher's observance of class schedule, evaluation practices, and instructional management. It asks the student to identify the teacher's strong points and aspects of the course which needs to be improved.

 

 

GENERAL GUIDELINES

1. The SET should be administered by a University Research Associate (URA) or a member of the administrative staff under the supervision of the Secretary of each College following the procedures described in this Student Evaluation of Teacher (SET) Manual of Administration.

2. The SET should be administered within the last three weeks of the semester, or on the last week of the Summer term.

3. The faculty/teacher being rated should not be present while the students fill out the SET.

4. SET data should be treated as confidential information. Access to these can be authorized only by the Dean, Secretary of the College, or Department Chair.

5. Collation of SET forms is to be done under the supervision of the College Secretary.

6. Data processing, collation, and analysis are to be done using the computer program furnished by the Office of the Director of Instruction (ODI) under the Office of the Vice-Chancellor for Academic Affairs (OVCAA).

 

 

SAMPLING

A 50 percent sample for each teacher will be randomly selected. However, for small classes of 15 or less, all students shall be included in the sampling.

If a faculty member teaches several sections of the same course, at least one section should be randomly selected by the College Secretary. If he teaches only one section, that class should be given the SET.

A 50 percent sample of the students in each class of the teacher to be evaluated will be selected by the University Research Assistant (URA).

To simplify the process, the URA starts from a random student and asks the other students to count off 1, 2, 1, 2, . , following a path around the room. To decide which students will answer the SET, he tosses a coin. If the coin turns up head, the 1's will be asked to stay and answer the SET. The 2's will be advised to leave the room. If the coin turns up tail, the 2's will answer the form.

 

 

METHOD OF ADMINISTRATION

Methods

1. The College announces the inclusive administration dates for the SET and picks the names of the URA/staff who will administer it.

2. The URA/staff administering the SET should check with the faculty member to make sure that the class will meet on the target date.

3. On the scheduled time, the assigned URA/staff goes to the room where the class meets within the last 30 minutes of the class schedule. The URA enters the room and introduces himself to the teacher. In no case should the visit be a surprise to the teacher. The URA then gives the teacher time to put a proper closure to the activity going on. On cue from the teacher, the URA says,

Good morning/afternoon/evening, class.

The teacher leaves the room after introducing the URA to the class.

Today, you will be answering a questionnaire called the Student Evaluation of Teacher (or the SET). This is a questionnaire which aims to help improve the quality of teaching. It is also a way of helping the faculty see how students perceive their teachers - specifically their style and management of instruction.

Some of you have experienced answering the SET. This time, I would like you to think back on this course [Give the course number and title] as you experienced it, how it was taught, and how well it has met your expectations.

I'm sure that by now you are already familiar with how the teacher taught the course to your class. Allow yourself a few minutes to reflect on the course, or review the syllabus.

Think particularly of your participation in it. Has the course met your expectations?

Recall how the teacher taught it and your particular satisfactions and dissatisfactions.

The URA sees that all count off 1, 2, 1, 2, . . When all have counted off, she says,

We will toss a coin to see who will answer the form. If the coin turns up head, the 1's are it. If tail, the 2's are it.

She tosses the coin. If tail, she says:

All 1's please pick up your things and quietly leave the room. You are dismissed. The 2's will stay and fill out the SET.

The URA waits and makes sure that everything is in order. Then she says to the remaining half:

Any questions ?
I will now distribute the forms to each of you.
The questionnaire has 4 pages. There are specific instructions on how to answer each part. Are you ready ?
 

[DISTRIBUTE THE MATERIALS]

You have 20 minutes to answer the questionnaire.
Hand me the form after you have reviewed and answered each item. You may now begin !
 

[ PAUSE ]

As soon as the students finish, or after 20 minutes, collect the materials. For those who submit the form early, take the time to skim through the pages to make sure that they answered all the items.

Thank the students for the time they gave to answer the questionnaire. Then, dismiss the class.

Bundle the materials when all have handed in the forms. Place them inside an envelope for submission to the Secretary's Office.

Scoring and Encoding

At least one URA/staff should be assigned to encode the data using the template/program provided by the ODI.

All responses should be encoded. A program has been prepared for this purpose. The answers to the open-ended responses should be collated for each teacher.

Responses to Items 1-26 in Part III are coded as follows:

 

Strongly Agree (SA) - 1
Agree (A) - 2
Disagree (D) - 3
Strongly Disagree (SD) - 4
Not Applicable (NA) - 9

Note the reverse scoring for these negatively oriented items in Part III-A: Items 10, 17, 19, 23, and 25.

Analysis for Each Teacher

a. Frequencies and percent distributions of responses for all I items
b. Straight listing of responses to open-ended questions
c. Mean ratings and standard deviations for items in Part III
d. Mean weighted ratings for items in Part III for all the sample classes of each teacher

Analysis for Each Course

a. The same as for each teacher
b. Frequencies for Parts I & II Multiple-Choice items

Format of Reporting

Prepare a feedback sheet for each teacher on these items:

Part I Mean ratings per item, 1-9
Part II A Mean ratings per item, 1-7
Part II-B Frequencies of responses for Items 1, 1.1, 2, 3, and 4
Part III-A Mean rating per item, 1-26 and overall mean rating
Part III-B Frequencies per item, 1-7

For uniformity and expediency, the excel program should be used following this forma once data encoded for submission to the ODI.

Faculty Name Course Title Class Size Course Type SET SCORES
No. of Students No. of Respondents Lab Lect. Part I Part II Part III Part IV
k k k k k k k k k k
k k k k k k k k k k
k k k k k k k k k k

Interpretation of Scores

The overall mean rating in Part III-A will be interpreted as follows:

1.00 - 1.24 Outstanding
1.25 - 1.75 Very Good
1.76 - 2.25
Average 2.26 - 2.75
Fair Below 2.75 Needs improvement

Reporting of Results

Feedback of SET results to the teacher concerned may be done through

a) the feedback sheets
b) a one-on-one talk
c) announcement of ratings of faculty (identities coded) as the Department Chair sees fit.

The summary of results should be forwarded to the ODI in the form suggested by that Office.

After encoding the data, the accomplished SET forms may be disposed by shredding or recycling after a period of three semesters. Until then, the Secretary's Office of each College will decide on where they are to be stored.

The Office of the Secretary will keep a copy of the diskette of the encoded data after sending one diskette copy to the ODI.

The Dean, the College Secretary and the Department Chair are the only staff who have access to the data. A faculty member may request access to his/her SET results through any of the three.

Suggested Uses of the SET Results

1. As basis for

1.1 promotion
1.2 identification of excellent teachers
1.3 grant of professorial chairs, incentive grants, etc.
1.4 tenure
1.5 reappointment

2. Information and feedback on

2.1 faculty strengths and weaknesses
2.2 skills to be addressed through in-service training
2.3 student-faculty relation
2.4 evaluation practices

3. Monitoring of observance of University rules (on absences, tardiness, consultation hours, etc.)

4. Identification of issues relative to

4.1 instruction
4.2 faculty behavior
4.3 classroom/school management
4.4 evaluation of teachers

 

 

ACKNOWLEDGMENTS

Original conceptualization of this manual was done in 1999 by a committee created by the Office of the Director of Instruction (ODI) under the Office of the Vice-Chancellor for Academic Affairs (OVCAA).

The committee members were:

Dr. Milagros D. Ibe (Chair)
College of Education

Prof. Therese Ann G. Capistrano 
School of Statistics

Mr. Angelico Clerigo
USC Representative

Dr. Jonathan Winston L. Salvacion
College of Engineering

Dr. Ana Maria G. Tabunda
School of Statistics

Dr. Aurora C. Zuniga
Program Development Associate Office of the Director of Instruction

Dr. Evelina M. Vicencio
Director Office of the Director of Instruction

Dr. Corazon M. Raymundo
Vice Chancellor Office of the Vice Chancellor for Academic Affairs

This manual was revised for distribution during the 1st Orientation for the Staff to Administer SET held on September 7, 2000 at the Benitez Theater of the U.P. College of Education. This was done under the new supervision of: Dr. Amelia P. Guevara Vice-Chancellor for Academic Affairs Dr. Celia T. Adriano Director, Office of the Director of Instruction Prof. Villy Ath. Buenaventura PDA, Office of the Director of Instruction

 

marvin nisperos 2004