I.  Introduction

This report by Policy Analysis for California Education (PACE) was commissioned by the University of California to inform deliberations of the Outreach Task Force on strategies to enhance University participation by students who are disadvantaged or from groups that have been historically underrepresented.  The report reviews evaluations of current outreach programs, identifies effective practices, and makes recommendations for the improvement of programs and of the methods used to evaluate programs.

Background
In July 1995 the Regents of the University of California adopted a new admissions policy abolishing consideration of race or ethnicity as a factor in admissions.  This policy reversed a long-standing affirmative action policy which included a students ethnicity as one factor in determining University competitiveness and selection. 

In February 1996 the University convened the Outreach Task Force to assist in developing strategies to maintain and enhance the participation of students who are disadvantaged in light of the new admissions policy.  In convening the Task Force, the University described three major aspects of their charge:  1) to review ways of increasing the means by which the University can enhance the eligibility rates of young people across the State, especially those from groups that have been underrepresented at the University; 2) to identify strategies to improve eligibility and participation of underrepresented students in the University; and, 3) to specify the principles or goals that the University should adopt to improve its outreach efforts.

The University of California has a long-standing commitment to the diverse population of this state.  As part of this commitment, the University also has a long history of involvement in pre-collegiate education, one form of which is "outreach"an array of programs and other activities designed to help prepare students so that they become competitive for, admitted to, and successful in the University.

Previous studies by the Office of the President have determined that low eligibility is the fundamental barrier to broader participation of disadvantaged students at the University.  A critical part of the work of the Task Force is to conduct a broad review of outreach programs and practices, at the University and elsewhere.  Outreach is the major way in which the University can make its campuses accessible to all qualified students.

As the Task Force got under way, its various working committees began to review an enormous amount of information pertaining to current outreach activities of the University.  The Task Force received copies of various documents, including The Schools and UC, a directory of more than 800 current programs that involve collaboration between the University and K-12 schools. 

At Task Force meetings, presentations were made on some of the universities largest outreach programs and on programs sponsored by individual campuses.  These presentations described programs with impressive results.  However, the information was difficult to interpret.  The sheer number of programs was bewildering, and little of the evaluation data were presented in written reports.  Many questions were raised about the rigor of the studies and the validity of the results.  Lacking more detailed information, how might the Task Force members determine which programs are more effective than others?  Is there a need for so many programs?  Are the programs efficiently managed and coordinated?  Would the students participating in these programs have qualified for the University without the program?

Purpose of this Report
In order to receive assistance in addressing these and other questions about outreach programs, the Task Force commissioned a study to review the evaluations of existing programs, both at the University and elsewhere.  The Task Force specifically wanted assistance in establishing criteria for efficient and effective programs and program elements and in determining the extent to which current systemwide UC programs meet criteria for efficiency and effectiveness.  In addition, the Task Force asked the evaluators to make recommendations for improving programs and for better evaluating present and future programs.

In commissioning this report to review current evaluations, the Task Force  also sought information that might inform deliberations on several policy issues regarding outreach program strategy:  1) the extent to which outreach programs should pursue directions that are tactical or strategic programs that provide special assistance to individual students versus programs that aim to enhance the overall capacity of K-12 schools; 2) the level of academic preparation that the programs aim for whether programs are aimed at helping students meet minimum eligibility criteria or at becoming competitive for admissions to the more selective campuses; and 3) the extent to which programs should be administered centrally or regionally. 

Scope and Procedures of the Review
The review of program evaluations included interviews and analyses of written documents.  Interviews were conducted with Outreach Task Force members, University administrative staff, outreach program directors and staff, and other individuals with a special perspective on the effectiveness of outreach programs.  Written documents reviewed included a variety of program reports and evaluations, meta-evaluations such as the two California Postsecondary Education Commission Reports on the Effectiveness of Intersegmental Student Preparation Programs, research and policy studies, and dissertations.

It is important to emphasize that this report is a synthesis of existing evaluations.  Given the time and resources available for this study, we did not, and could not, independently evaluate the programs.  Thus the scope of the review and analysis of findings is limited to the documentation that currently exists. 

A Critical Role for the University
The evidence reviewed suggests that outreach programs are beneficial to students.  However, we note at the outset that there is currently very little information that can really help policymakers make decisions about which programs are most effective and where scarce resources should be invested.     
Few program evaluations have been conducted rigorously.  No evaluations have systematically investigated the differential effects of one program component versus another.  Very little information is available to determine how many students might have attended college without the programs.  Very few evaluations follow students to document how they fare once they attend higher education institutions.

It is clear to us that the University must take responsibility for building a framework for coordinating and evaluating outreach efforts so that the programs can fit into an overall strategy.  However, it is noteworthy that this lack of attention to evaluation of outreach programs is by no means unique to programs of the University of California.  We found this same situation in our review of programs in other states.  It is unreasonable to expect the individual programs to independently improve the quality of service delivery and evaluation.  The program staff have neither the resources nor the perspective to bear primary responsibility for improving program accountability.  The responsibility for coordination and accountability must be assumed by an entity, such as the University, which has a stake in the success of the overall K-16 education system.   

Organization of the Report
Subsequent chapters of this report present an analysis of what has been learned about outreach programs in order to inform deliberations relative to the following questions:

1.  What do we know about the effectiveness of current efforts to increase the numbers of underrepresented and disadvantaged students who are well-prepared for higher education?  Are there certain practices or program components that are most effective in college preparation programs?
2.  How might the evaluations of outreach programs be improved?
3.  What essential principles should be considered in designing college preparation programs for disadvantaged students?
4.  What are the implications of these findings for policy decisions on strategies of future outreach efforts?

Chapter II of the report presents information on the flow of students in the K-16 education pipeline.  This information provides parameters on the problem of improving college preparation of disadvantaged students.  Also in Chapter II is a discussion of the barriers to University participation of disadvantaged students. 

Chapter III provides brief summaries of evaluation findings for examples of various types of programs University of California systemwide programs as well as other programs in California and nationally. 

In Chapter IV we discuss effective practices and suggest essential principles for designing student centered and school centered programs.

In Chapter V is a review of current program evaluation strategies with suggestions to improve future evaluations.  A more expanded discussion of  program evaluation designs is included in an appendix to the report.
Finally, in Chapter VI we discuss implications for the Universities outreach policies.

BACK  |  HOME  |  NEXT