Procedures Followed in This Review
Our review of the published research literature and of programs nationally suggests that there are remarkably few evaluation studies of outreach or college preparation programs that meet established standards of effective evaluation.  Researchers and higher education scholars throughout the country lament the paucity of good evaluation studies of these programs. Most of the data that exist are of a descriptive nature, and the data systems are not in place that would allow long-term tracking of students.  Nearly all studies which report student outcomes are limited to short-term outcomes (e.g., college eligibility) and do not include long-term outcomes (e.g., college graduation).  The programs themselves tended to be developed independently and without either the designs or the resources for systematic evaluation.  Thus, while most programs are able to show positive overall results, many questions are left unanswered.  Would the students have succeeded without the program?  What is it about the program that contributes to its apparent success?  Are different types of interventions more effective with different types of students? 

A subsequent chapter of this report provides a critique of the evaluation methods used in current programs and makes recommendations for future evaluations that would yield more evidence of program effects more useful for policy makers. 

Not withstanding the limited evaluation data, there is much that has been learned about outreach programs that can inform deliberations about future efforts.  A few programs have conducted quite extensive and sophisticated evaluations of program implementation, student outcomes, or both.  The experience and views of individuals who are knowledgeable about the programs is also of great value in evaluating the programs.  Thus, while we may not be able to draw conclusions with the level of confidence that we would like, we can certainly build on what we know now in planning future strategies.

Procedures followed in this review of the effectiveness of outreach programs included interviews and analyses of program documents and evaluation reports.  Interviews were conducted with Outreach Task Force members, University administrative staff, outreach program directors and staff, researchers, and other individuals with a special perspective on the effectiveness of outreach programs.  Written documents reviewed included a variety of program reports and evaluations, meta-evaluations, research and policy studies, and dissertations. (See Bibliography.)

Summaries of Program Evaluations
This section provides brief summaries of program evaluations.
The review begins with two reports by the California Postsecondary Education Commission (CPEC) which summarize the evaluations of nine intersegmental college preparation programs.  Following the CPEC reports we briefly present evaluation findings from a sample of programs that exemplify  the range of programs along a continuum from student-centered programs to school-centered programs or systemic initiatives.

The CPEC Reports on the Effectiveness of College Preparation Programs
The California Postsecondary Education Commission (CPEC) has published two noteworthy evaluation reports on the effectiveness of collaborative programs to prepare students for college.  A three-year study of nine programs was completed in 1992, and a follow-up study of eight of these programs and one additional program was completed in 1996.  In both reports CPEC concluded that the programs, individually and collectively, were highly effective and efficient.  The reports document high levels of eligibility and attendance at California's public universities for students served by these programs when compared to other underrepresented students or to graduating seniors generally.

The CPEC reports are based on information provided by the programs.  With some noteworthy exceptions, the data on student outcomes are mostly descriptive statistics without appropriate comparisons, and are limited to short-term outcomes. Thus, while the findings are strongly suggestive of positive outcomes for students, they unfortunately are not conclusive and provide little insight into what makes the programs effective.

We know from the most recent CPEC report that only 8.6 percent of underrepresented students statewide participated in the nine programs included in the report.  In addition, CPEC reports that only 7.5 percent of California elementary and secondary schools were involved in these programs.  The report makes a strong recommendation for additional resources to expand both the numbers of schools participating in these programs and the numbers of students served.

While there can be no argument that there is an acute need to increase efforts to prepare underrepresented students for college, the CPEC reports provide little basis for making decisions about how to go about expanding the efforts. These student-centered programs do not all provide the same services, and surely they do not all work equally well.  Unfortunately, we simply do not have the evaluative data that would enable us to systematically compare results across programs or across program components.

The CPEC reports provide some information on sources of funds for the programs.  From this information CPEC estimates program costs per student. They estimate an overall cost per student of  $140.21 for 1994-95.  It is helpful to have some cost information about the programs.  While it would be extremely useful if per student costs could be compared across programs, CPEC states and we agree that such comparisons would not be appropriate.  The programs differ on so many dimensions that meaningful cost comparisons cannot be made across programs.  We conclude further that current evaluation data for the programs included in the CPEC reports, as well as for most other programs do not enable us to make meaningful cost-effectiveness evaluations.     

Clusters of Programs
This section will characterize the range of college preparation programs and discuss those which are the focus of this review.  For ease of comparison we have organized this section into clusters, or groupings, of programs.  We have not included all of the programs we have reviewed such a list would be extensive but would provide little additional information.  We have selected those programs which are most prominent, or which have evaluation data, or which represent features we think are noteworthy.  In conducting the review, we examined UC programs, programs administered by other education segments within California, and programs in other states.

The distinctions between the clusters is admittedly blurred, none of the programs represent pure models.  The four clusters are:

1. Programs which are primarily student-centered.  These programs represent efforts to provide services directly to students.  They commonly feature activities which promote academic enrichment and support, parental involvement, counseling, advisement, information on college admissions, and motivation. 


2. Programs which combine student-centered approaches with enhanced student financial aid.  These programs provide the link between student-centered activities and financial assistance.


3. Programs which combine student-centered and school-centered strategies.  These programs combine direct student services with staff development and other activities to improve the overall quality of curriculum and instruction in the school.

BACK  | HOME  |  NEXT