A Framework for Program Evaluation
The essential underpinning of evaluation of the universities outreach programs is a University-wide longitudinal student information system.  We suggest that the University consider developing a pre-collegiate information system for its outreach programs so that longitudinal data will be available to track students participating in outreach programs while they are still in K-12 and continuing after they matriculate at the University.

We understand that the MESA staff are currently working on the design of a student information system which could become a prototype for a more comprehensive system.  A student information system, in our view, is the most immediate action that should be taken to strengthen evaluation and could go a long way toward removing the uncertainties surrounding current data which rely on Ahead counts with unknown reliability.

The presidents Office must be specific about what it expects as benchmarks against which to measure progress.  A uniform way of reporting descriptions of program components, performance indicators and most importantly, the program goals underlying the particular approach is also required.  As we outlined in Chapter V, what occurs in most of existing UC outreach evaluations is a failure to describe adequately the program and its components and to link program activities with performance indicators.  This makes it virtually impossible to distinguish outcomes between programs and, within programs, to ascertain which components are more or less effective in reaching the intended goals.  It makes it impossible to do any cost- effectiveness analysis.  Absent a uniform way of presenting basic information, comparative analyses are fruitless.

In most instances, staff of specific programs although they believe in what they are doing and are dedicated to delivering services lack the resources, the incentives, and the perspective to assume primary responsibility for evaluating their programs.

The Office of the President should support campus efforts to systematically evaluate their programs.  At the system level, evaluation and research efforts should focus on filling in the gaps and conducting research that local programs are unlikely, or unable to undertake.  It is the responsibility of the presidents Office to provide additional technical assistance and professional development opportunities to assist program directors in these efforts.¹⁵

Another essential element in the framework is a systematic research agenda which sets aside sufficient funds to address specific research questions.  Given the program diversity and complexity, research efforts will require a multi-level, multi-stage approach.  We outline in Chapter V, five types of research approaches which need to be supported by the University.  It is important to have a multiple approach strategy within which to build a portfolio of data, specifically designed to inform policy makers about options and to determine the most strategic ways to invest scarce resources.

The research agenda is further complicated by the very nature of these programs in most cases they involve K-12 schools as partners.  The University must be sensitive to the schools accountability needs as well and  work closely and cooperatively with them in building a powerful research agenda.

Conclusion
The  Outreach Task Force will help chart the course for maintaining and enhancing diversity at the University of California in light of new admissions policies.  As a highly selective institution, the University must employ multiple strategies.  First, it must enhance recruitment to increase the number of  eligible and competitive underrepresented students who choose to attend the University.  Second, it must aggressively pursue efforts to help prepare larger numbers of students who are reasonably close to meeting admissions standards.  Third, it must devote resources to a strengthening of the K-12 education pipeline.  Pursuing this third strategy requires that the University become a full partner with its higher-education and K-12 colleagues.

BACK  |  HOME  |  NEXT