WASC Proposal

This is a draft of the proposal that will be sent to David Goodstein for inclusion in the overall WASC proposal from Caltech. Our part should be between 2 and 3 pages. This text can be editted by logging on to the wiki and clicking the edit button.

Updates

• 24 Aug 06: updated proposal to align with WASC Handbook, pages 39-40. Comments regarding linkages to the handbook are in green.
• 30 Apr 06: cut down to two topics and focused those topics a bit. This is the final version that was used as input to the overall WASC Proposal.
• 18 Apr 06: added third topic (mentoring) plus various edits based on CCSC meeting. Changes are marked in blue, comments are marked in green.
• 16 Apr 06: first draft

---

Undergraduate Education

Andrea Dubin (ARC), Marianne Bronner-Fraser (BIO), Jean Ensminger (HSS), Csilla Felsen (ARC),
Brad Filippone (PMA), Nate Lewis (CCE), Richard Murray (chair), Niles Pierce (EAS), Josephine Suh (ARC)

Overview of Caltech Undergraduate Educational Program

Institutional context: A Statement of Institutional Context that briefly describes the institution’s background, its current context, a brief and basic presentation of its major strengths and challenges, and a synopsis of responses to issues raised by the Commission action letter and, where relevant, issues identified by the Substantive Change or Interim Report Committees. This Statement should also connect the context for the accreditation review to the institution’s vision and strategic plan.

Caltech offers a four-year undergraduate course with "options" (majors) available in applied and computational mathematics; applied physics; astrophysics; biology; business economics and management; chemical engineering; chemistry; computer science; economics; electrical engineering; engineering and applied science; English; geobiology; geochemistry; geology; geophysics; history; history and philosophy of science; independent studies; mathematics; mechanical engineering; philosophy; physics; planetary science; and social science. Each leads to the degree of Bachelor of Science.

Option requirements are established by the faculty responsible for the given degree program and consist of a set of required courses as well as recommended couses. Each course at the Institute is assigned a number of units corresponding to the total number of hours per week devoted to that subject, including classwork, laboratory, and the normal outside preparation. The Institute requirement for a Bachelor of Science degree is 486 units, which corresponds to approximately 40 units per quarter for a period of four years.

All options require students to take courses in biology (6 units), chemistry (24 units), humanities (54 units), mathematics (45 units), physics (45 units), and the social sciences (54 units). These courses form the "core curriculum", which is required of all students regardless of their degree. Course work is rigorous and students are encouraged to participate in research. The undergraduate program is thus designed to provide an intensive exposure to a wide spectrum of intellectual pursuits.

In the first year of study, students are assigned faculty advisors based on their very rough areas of interest (engineering, physics, etc). Near the end of the first year, students select an option, and during the second year they begin to specialize. Each student is then assigned a faculty advisor within the option that they select. This is normally done without regard to a students special interests (beyond the option specialization), but students can request a change of advisor if they desire. Although the average advising load across Caltech is approximately 3-4 students per faculty member, in some of the more popular options the advising load can be as high as 10-12 students per faculty member (excluding freshmen advisees).

Caltech also encourages a reasonable participation in extracurricular activities, which are largely managed by the students themselves. Three terms of physical education are required, and intercollegiate and intramural sports are encouraged. In short, every effort is made to provide undergraduate students with well-rounded, integrated programs that will not only give them sound training in their professional fields, but that will also develop character, intellectual breadth, and physical well-being.

Teaching assignments and course evaluations are the responsibility of the individual Divisions at Caltech. Most Divisions delegate the responsibility for teaching assignments to the Executive Officer for each option, who is the faculty member responsible for the undergraduate (and also graduate) degree program in a given area. Teaching evaluations are handed out in all undergraduate courses and are returned to the Division Chair, who is responsible for faculty evaluation and raises. A new online course survey is currently being developed for courses that are part of the core curriclum, which provides more detailed feedback to instructors.

Issues for Further Study

Based on consultation with faculty, students and staff over the course of the past 6 months, the committee identified many potential areas for study. To help insure that we are able to make positive progress, we have identified two specific areas where we plan to focus our efforts. In each of these areas, we feel that a thorough assessment of Caltech's educational program would be timely and could lead to constructive changes in the undergraduate program.

Teaching Quality

While there are many excellent teachers at Caltech, there are also examples of courses that are poorly taught and instructors who could do a much better job in their teaching performance. At the present time, there are relatively few resources available for Caltech faculty who want to improve their teaching and limited feedback mechanisms for improving the teaching effectiveness. In our initial discussions, we have identified several aspects of teaching quality that we believe should be further studied:

Course Feedback - How do faculty get feedback on their courses, both during the term and after the term? Can course surveys, web sites, student lunches and other mechanisms be used to provide instructors more information about whether the students are understanding the material and whether the teaching methods are effective?

Teacher Training - How can Caltech usefully provide information to teachers about best practices for effective teaching, available resources and technologies, and integration of the honor system into their courses?

Faculty Incentives - Traditional teaching incentives (awards, salary increases) typically reward the best teachers but may not influence the bulk distribution of teaching. Are there incentives that can be provided to increase the average quality of teaching across the campus? Are there disincentives that can be provided to motivate those faculty that intentionally neglect their teaching responsibilities? How can faculty be assigned to the types of courses that best fit their style and abilities?

New Methods - How might Caltech explore new methods of teaching, such as active learning, lecture-less classes and project-based courses? What new technologies might we consider for incorporation into courses and how do we provide information about these technologies to interested faculty.

Undergraduate Workload

Another common area of concern at Caltech is the workload of the undergraduates. On the one hand, many Caltech courses are notorious for requiring more work than the number of units would indicate. At the same time, some Caltech undergraduates take a large number of units each term, creating a situation in which they have little time to integrate the material they are learning or pursue activities outside of class (including research opportunities at Caltech). Some possible aspects for further study are:

Course Underuniting - how can the number of units required for a course be accurately determined and assigned to each course? Will students be able to finish their graduation requirements if the units are increased appropriately?

Work Schedule - the timing of homework, labs, exams and courses can often lead to situations in which students must choose between completing their work and attending classes and labs. Can methods for resolving these timing conflicts be identified and implemented?

Class Attendance - in some courses, the attendance in class is lower than 50%. Are there reasons for this lack of attendance and should lectures be restructured to encourage greater attendance?

'Student Health - how is the Caltech workload affecting student health (mental and physical)? Are there changes that could be made that would provide less stress on students while maintaining Caltech's rigorous educational program?

Proposed Approach

The two topics above represent important areas for the undergraduate educational program at Caltech and are deserving of further study. To help focus our efforts, we plan to collect substantial data that helps identify those areas where further study and changes would have the most impact. In particular, it is important to recognize that many aspects of Caltech's educational program work very well and we want to be careful not to make changes for the sake of change, but rather to be driven by a thorough understanding of what problems we are trying to solve and what metrics we will use to judge success over time.

We plan to conduct our self study in four phases:

• Phase I (Oct 06 to Mar 07) - Data Collection
• Phase II (Apr 07 to Dec 07) - Campus Discussion
• Phase III (Jan 08 to Jun 08) - Findings and Recommendations
• Phase IV (Oct 07 to Jun 08) - Initial Implementation

Data Collection

We plan to spend the first part of the study period collecting substantial data regarding the current state of undergraduate education at Caltech as well as benchmarking against other universities with missions similar to Caltech. These activities will be undertaken by the WASC undergraduate education committee, which consists of faculty from a variety of departments as well as undergraduate students. Our current plan is to carry out these activities in the fall and winter quarters of the 2006-07 academic year, which will lead into the student/faculty conference planned for April 2007 (described in the next section).

We plan to use the following mechanisms for collecting data:

Collect information through survey data, polling people on campus (both faculty and students). Present the data to faculty and students to generate more discussion and come up with solutions. Have committee on Undergraduate Education in the upcoming Student Faculty Conference to make this a more broad Caltech campus discussion.

Class surveys - Caltech surveys incoming freshman and outgoing seniors each year. The surveys for 2006 have already been updated to include questions on some of the issues raised above and we can add additional questions to the 2007 and 2008 surveys as the self study progresses.

Course surveys - In addition to surveys of an entire undergraduate class, Caltech also has course evaluations for individual courses. A new online course evaluation is being used for the core curriculum and this can be used to collect data regarding to those specific courses. Data from both the online surveys and the paper surveys will be available to the committee.

Requirements analysis - To understand student workload issues, course scheduling and related issues, we will work with the registrar's office to analyze the courses taken by Caltech undergraduates over the past several years.

Benchmarking - We plan to investigate techniques used at other schools to provide benchmarks for our analysis. We plan to identify 2-3 schools for doing a detailed benchmarking study. Candidate schools include MIT, Harvey Mudd College and Stanford.

Existing data - Caltech employs several mechanisms for feedback on its education and research program, including ABET accreditation for its engineering curriculum, external visiting committees for each Division, and bi-annual student faculty conferences. In many cases, information on undergraduate education has been collected and discussed in these forums and we will make use of this collective wisdom.

Campus Discussion

The second phase of our self study will be to discuss the data that we have collected with students and faculty, and decide on what specific topics to pursue for improving the quality of undergraduate education at Caltech. This discussion will begin with the bi-annual Student/Faculty Conference, currently scheduled for April 2007. This conference is organized by the students and we intend to form a committee on undergraduate education whose goals will align closely with those of the WASC undergraduate education committee. The output of the Student/Faculty conference is a report that includes recommendations of areas for further improvement, which will be incorporated into our self-study findings and recommendations.

In addition to the Student/Faculty Conference, we intend to use the second phase of the self study to discuss our data analysis and preliminary findings with several other groups on campus. These include:

• Academic Policies Committee - The Academic Policies Committee is responsible for making a continuous study of the Institute's academic policies. It is not be limited in any way concerning the subjects that it may take under consideration for discussion and recommendation to the Faculty Board.

• Academics and Research Committeee - This undergraduate committee serves as an objective liason between students and faculty, to facilitate effective communication, and improve the quality of learning at Caltech.

• Core Curriculum Steering Committee (CCSC) - The CCSC coordinates and supervises content and teaching of the Core Curriculum. The Committee also monitors the performance of the Core Curriculum courses and devises improvements in the core. Any proposed change in Core Curriculum courses must be reviewed and approved by the the CCSC before it is considered by the Curriculum Committee and the Faculty Board.

• Curriculum Committee - The Curriculum Comittee is responsible for general supervision over the undergraduate curriculum. All proposed changes in the undergraduate program and all proposed changes in courses, including the presentation of new courses that undergraduate students will normally take, are considered by this committee for recommendation to the Faculty Board.

• Institute Academic Council (IACC) - The IACC consists of the Division Chairs of Caltech's six academic divisions along with the President and Provost. This committee is ultimately responsible for adminsitering all teaching activities at Caltech.

• Undergraduate Academic Standards and Honors (UASH) Committee - this faculty committee is responsible for awarding a variety of academic honorsand for ensuring that a consistent set of standards and rules is applied in academic matters involving undergraduates.

Findings and Recommendations

Based on the data collected in Phase I and the discussions with the various committees and campus constituencies in phase II, the final phase of our self study will be to formulate finding and recommendations. These findings and recommendations will be provided to the Caltech administration and the Faculty Board for discussion and implementation. The timing of this phase is chosen to allow ample discussion during the academic year, so that recommendations can be acted upon in the 2008-09 academic year.

Initial Implementation

Initial implementation of recommendations generated by the committee will occur in the 2008-2009 academic year and can be assessed at the end of that year to measure whether they are making progress.

---

Unused Material

The material below was removed from the proposal to keep it concise and focused.

Teaching Quality

• Role of the Core Curriculum - The core curriculum is the dominant experience for students arriving at Caltech and establishes the common background for option-specific courses. Is the teaching in the core preparing students for the style of learning that they will encounter in later classes? Are core courses providing the material required for later courses and for students' post-Caltech education and careers? [RMM: Scott Fraser would really like us to include some evaluation of the core, not necessarily to change it, but to provide data about its relevancy. Should we include this here?]

Student Workload

• Independent Study - is the structure of the Caltech undergraduate education one that allows "inspired independent study" for those students who are interested? How might research opportunities be better integrated into the curriculum requirements. [RMM: This topic overlaps with the Undergraduate Research self-study topic.]

Student Advising

[RMM: This came up in the CCSC meeting and there was a lot of resonance with looking into this.]

The low student faculty ratio at Caltech (approximately 3:1) allows undergraduates to have a much richer interaction with their faculty advisors than might be possible at a larger school. However, there are challenges within the structure of Caltech that can lead to less than idea situations. One issue is that students are not uniformly distributed across Caltech's departments, so while the student/faculty ratio might be low in some areas, it can be quite high in others (e.g., in recent years the EE faculty have averaged 12 student advisees per faculty member, not including freshmen). In addition, the intense research focus at Caltech provides opportunities for some students to get involved a faculty lab during their studies, but some students complain that faculty to not spend adequate time on their advising activities. Some possible areas of study include:

• Advising Load - How can advising be better distributed across the entire Caltech faculty, so that we can take best advantage of the low student/faculty ratio at Caltech?

• Advisor Training - What information and training should Caltech provide to faculty advisors? How might best practices in undergraduate advising be shared between faculty?

• Advisor Assignments - How should students be matched with an advisor? Should advisor assignments remain static during in the sophomore through senior years?

• Advisor Feedback - How can students provide constructive feedback to their advisors, especially given the very individual (and hence attributable) nature of the advisor/student relationship?

Other Issues

[RMM: I want to eliminate this section. If there are topics here that are really important, we should try to include them under the three board themes above.]

In addition to these broad issues, there are a number of more specific items that we would like to collect information about and consider as possible components of the self study on undergraduate education:

• Classroom experience (across the variety of courses that students take)
• Might touch on tools (eg, MATLAB), breadth of the core [RMM: captured under core evaluation]
• Look into student health? Compare to other universities [RMM: captured under student workload]
• Perhaps link to research and honor system topics
• Course attendance, classroom experience and *learning* [RMM: captured under teaching quality]
• Can we find out why students aren't coming to class? [RMM: captured under student workload]
• Focus on what students learn?
• Content of the core [RMM: captured under core evaluation]