he primary rationale for using standardized tests, such as the SAT, in college admissions is to predict success in college. Quoting from a recent publication of the College Board,

The SAT has proven to be an important predictor of success in college. Its validity as a predictor of success in college has been demonstrated through hundreds of validity studies. These validity studies consistently find that high school grades and SAT scores together are substantial and significant predictors of achievement in college.¹

Yet while it is true that the "predictive validity" of the SAT I has been widely studied, the same cannot be said of the SAT II achievement tests, which have been relatively ignored. One reason for that neglect is that very few colleges and universities require the SAT II -- the University of California being the largest and most notable exception. In fact, UC has required applicants to submit both SAT I (or ACT) scores and SAT II achievement test scores since 1968. As a result, UC has amassed an extensive database on the two tests, and we are uniquely positioned to assess their relative utility in predicting success in college.

This paper presents preliminary findings on the relative contribution of high-school grade-point average (HSGPA), SAT I and SAT II scores in predicting college success for 81,722 first-time freshmen who entered UC over the past four years, from Fall 1996 through Fall 1999, inclusive. The criterion of collegiate "success" employed here is the same as that used by the College Board in the majority of its research on the SAT - freshman GPA. Quoting again from the College Board:

The overwhelming majority of these studies use … freshman GPA as the criterion representing success in college. Freshman GPA is the most frequently used criterion because:
• The courses that freshmen take are more similar and less variable than at any other year in college, thus minimizing comparability issues that occur with grades;
• Predictor and criterion data are readily available; and

• Freshmen grade averages are highly correlated with cumulative grade averages.²

Many have criticized the narrowness of freshman GPA as a measure of success in college and have urged that other criteria, such as college graduation rates, be used instead. At the request of BOARS, UCOP researchers are examining the relationship between SAT scores and persistence and graduation rates at UC, and those findings will be presented in a later analysis. For purposes of this analysis, however, we have chosen to focus on UC first-year GPA (UCGPA), since freshman GPA is by far the most commonly employed measure in studies of the predictive validity of college admissions tests, and because use of the SAT is most often justified on this basis.

Preliminary Findings

The table below shows the explained variance³ (also denoted "R-Square") in first-year UCGPA that is accounted for by various predictor variables. In this case, three predictor variables were studied - HSGPA, SAT I and SAT II composite scores -- for all freshmen entering UC in Fall 1996 through Fall 1999. The effects of the predictor variables on UCGPA were analyzed both singly and in combination, as displayed below:

Review of these data suggests three main conclusions:

• First, looking at the predictor variables one by one - rows (1) through (3) in the table above -- SAT II scores were the best single predictor of UCGPA in two of the four years studied (1998 and 1999), and also the best single predictor for the pooled, 4-year data. Over the four-year period, SAT II scores accounted for the most variance in UCGPA, 15.3%, followed by HSGPA with 14.5%. SAT I scores ranked third, accounting for 12.8% of the variance in UCGPA in a single-variable prediction equation.
• Second, using the predictor variables in combination - rows (4) through (7) in the preceding table - the proportion of explained variance increases beyond that which is possible using any one variable alone. Thus, the three predictor variables combined - HSGPA, SAT I and SAT II (row 7) - account for 21.1% of the total variance in UCGPA over the past four years (row 7, right-hand column).⁵
• Third and finally, it is evident that SAT I scores add very little, if any, incremental power in predicting UC freshman grades after SAT II scores and HSGPA are taken into account. SAT II scores and HSGPA together account for 21.0% of the variance in UCGPA in the pooled, 4-year data (row 6, right-hand column). Adding SAT I into the equation (row 7) improves the prediction by an increment of only 0.1% in the pooled, 4-year data. Indeed, in two of the four years (1997 and 1998), SAT I scores add nothing to the explained variance.

These findings are necessarily preliminary. The President is taking steps to support independent UC faculty research on standardized tests, and under the direction of BOARS, UCOP research staff is conducting in-depth analyses of the validity and impact of the SAT I, SAT II and ACT. UCOP has also contracted with the National Center for Research on Evaluation Standards and Student Testing to assess the potential of alternative tests, such as the Golden State Examinations, for use in UC admissions.

In summary, analysis of the performance of over 81,000 students who entered UC during the past four years suggests strongly that SAT II composite scores and high school grades together are the best predictors of freshman grades -- the standard measure of collegiate "success" employed by the College Board in predictive validity studies - and that SAT I scores add little, if any, incremental value to the prediction.

1. Camara, W.J. and Echternacht, G. (2000). "The SAT I and High School Grades: Utility in Predicting Success in College," The College Board: Office of Research and Development, New York: NY, p. 9.

2. Ibid., p. 1.

3. For those unfamiliar with the terminology of predictive validity studies, explained variance, also known as the coefficient of determination or R2, represents the proportion of total variance in an outcome variable, such as UCGPA, that is accounted for or "explained" by a predictor variable, such as HSGPA or SAT scores. Explained variance ranges from 0 to 1, and can also be expressed as a percentage. For example, in the table above, in 1996 HSGPA accounted for 0.170, or 17%, of the total variance in UCGPA.

4. Under current UC policy on eligibility for admissions, scores on different tests or sub-tests are summed to produce an overall composite score. For the SAT I, the math and verbal sections are summed to produce an SAT I composite score; for the SAT II, students are required to take two tests - Writing and Mathematics Level 1 or Level 2 - plus a third subject test of the student's choosing, and scores on the three tests are summed. The maximum possible composite score is 1600 on the SAT I, and 2400 on the SAT II. HSGPA is honors-weighted GPA with additional grade-points for honors-level courses; HSGPA is uncapped and may exceed 4.0.

5. To those unfamiliar with predictive validity studies, the fact that HSGPA, SAT I and SAT II scores account for only about a fifth of the total variance in UCGPA - leaving almost four-fifths unexplained - may seem odd, but this relatively low level of predictive power tends to be the norm in admissions research. One of the reasons for the relatively low predictive power of standardized tests and high school grades is a problem known as "restriction of the range," that is, the fact that students with low test scores and grades often do not apply to selective institutions, and among students who do apply, only those with higher test scores and grades tend to be admitted. The result is that nearly all admitted students at selective institutions tend to have high test scores and grades, and there is not a broad enough range of students with which fully to assess the predictive validity of these admissions criteria. Some researchers advocate "correcting" or "adjusting" prediction estimates to account for restriction of the range, but the adjustments are not straightforward and depend in part on the assumptions of the researcher. For that reason, only observed or "uncorrected" statistical relationships have been presented in this preliminary analysis, although the "restriction of the range" issue will be examined in subsequent research.