primary rationale for using standardized tests, such as the SAT,
in college admissions is to predict success in college. Quoting
from a recent publication of the College Board,
SAT has proven to be an important predictor of success in college.
Its validity as a predictor of success in college has been demonstrated
through hundreds of validity studies. These validity studies consistently
find that high school grades and SAT scores together are substantial
and significant predictors of achievement in college.1
Yet while it is true that the "predictive validity"
of the SAT I has been widely studied, the same cannot be said of
the SAT II achievement tests, which have been relatively ignored.
One reason for that neglect is that very few colleges and universities
require the SAT II -- the University of California being the largest
and most notable exception. In fact, UC has required applicants
to submit both SAT I (or ACT) scores and SAT II achievement test
scores since 1968. As a result, UC has amassed an extensive database
on the two tests, and we are uniquely positioned to assess their
relative utility in predicting success in college.
This paper presents preliminary findings on the
relative contribution of high-school grade-point average (HSGPA),
SAT I and SAT II scores in predicting college success for 81,722
first-time freshmen who entered UC over the past four years, from
Fall 1996 through Fall 1999, inclusive. The criterion of collegiate
"success" employed here is the same as that used by the
College Board in the majority of its research on the SAT - freshman
GPA. Quoting again from the College Board:
Many have criticized the narrowness of freshman
GPA as a measure of success in college and have urged that other
criteria, such as college graduation rates, be used instead. At
the request of BOARS, UCOP researchers are examining the relationship
between SAT scores and persistence and graduation rates at UC, and
those findings will be presented in a later analysis. For purposes
of this analysis, however, we have chosen to focus on UC first-year
GPA (UCGPA), since freshman GPA is by far the most commonly employed
measure in studies of the predictive validity of college admissions
tests, and because use of the SAT is most often justified on this
The table below shows the explained variance3
(also denoted "R-Square") in first-year UCGPA that is
accounted for by various predictor variables. In this case, three
predictor variables were studied - HSGPA, SAT I and SAT II composite
scores -- for all freshmen entering UC in Fall 1996 through Fall
1999. The effects of the predictor variables on UCGPA were analyzed
both singly and in combination, as displayed below:
of these data suggests three main conclusions:
- First, looking at the predictor variables one
by one - rows (1) through (3) in the table above -- SAT II scores
were the best single predictor of UCGPA in two of the four
years studied (1998 and 1999), and also the best single predictor
for the pooled, 4-year data. Over the four-year period, SAT II
scores accounted for the most variance in UCGPA, 15.3%, followed
by HSGPA with 14.5%. SAT I scores ranked third, accounting for
12.8% of the variance in UCGPA in a single-variable prediction
- Second, using the predictor variables in combination
- rows (4) through (7) in the preceding table - the proportion
of explained variance increases beyond that which is possible
using any one variable alone. Thus, the three predictor variables
combined - HSGPA, SAT I and SAT II (row 7) - account for 21.1%
of the total variance in UCGPA over the past four years (row 7,
- Third and finally, it is evident that SAT I scores
add very little, if any, incremental power in predicting UC freshman
grades after SAT II scores and HSGPA are taken into account.
SAT II scores and HSGPA together account for 21.0% of the variance
in UCGPA in the pooled, 4-year data (row 6, right-hand column).
Adding SAT I into the equation (row 7) improves the prediction
by an increment of only 0.1% in the pooled, 4-year data. Indeed,
in two of the four years (1997 and 1998), SAT I scores add nothing
to the explained variance.
These findings are necessarily preliminary. The
President is taking steps to support independent UC faculty research
on standardized tests, and under the direction of BOARS, UCOP research
staff is conducting in-depth analyses of the validity and impact
of the SAT I, SAT II and ACT. UCOP has also contracted with the
National Center for Research on Evaluation Standards and Student
Testing to assess the potential of alternative tests, such as the
Golden State Examinations, for use in UC admissions.
In summary, analysis of the performance of
over 81,000 students who entered UC during the past four years suggests
strongly that SAT II composite scores and high school grades together
are the best predictors of freshman grades -- the standard measure
of collegiate "success" employed by the College Board
in predictive validity studies - and that SAT I scores add little,
if any, incremental value to the prediction.
1. Camara, W.J. and Echternacht, G. (2000). "The
SAT I and High School Grades: Utility in Predicting Success in College,"
The College Board: Office of Research and Development, New York:
NY, p. 9.
2. Ibid., p. 1.
3. For those unfamiliar with the terminology of predictive validity
studies, explained variance, also known as the coefficient of determination
or R2, represents the proportion of total variance in an outcome
variable, such as UCGPA, that is accounted for or "explained"
by a predictor variable, such as HSGPA or SAT scores. Explained
variance ranges from 0 to 1, and can also be expressed as a percentage.
For example, in the table above, in 1996 HSGPA accounted for 0.170,
or 17%, of the total variance in UCGPA.
4. Under current UC policy on eligibility for admissions, scores
on different tests or sub-tests are summed to produce an overall
composite score. For the SAT I, the math and verbal sections are
summed to produce an SAT I composite score; for the SAT II, students
are required to take two tests - Writing and Mathematics Level 1
or Level 2 - plus a third subject test of the student's choosing,
and scores on the three tests are summed. The maximum possible composite
score is 1600 on the SAT I, and 2400 on the SAT II. HSGPA is honors-weighted
GPA with additional grade-points for honors-level courses; HSGPA
is uncapped and may exceed 4.0.
5. To those unfamiliar with predictive validity
studies, the fact that HSGPA, SAT I and SAT II scores account for
only about a fifth of the total variance in UCGPA - leaving almost
four-fifths unexplained - may seem odd, but this relatively low
level of predictive power tends to be the norm in admissions research.
One of the reasons for the relatively low predictive power of standardized
tests and high school grades is a problem known as "restriction
of the range," that is, the fact that students with low test
scores and grades often do not apply to selective institutions,
and among students who do apply, only those with higher test scores
and grades tend to be admitted. The result is that nearly all admitted
students at selective institutions tend to have high test scores
and grades, and there is not a broad enough range of students with
which fully to assess the predictive validity of these admissions
criteria. Some researchers advocate "correcting" or "adjusting"
prediction estimates to account for restriction of the range, but
the adjustments are not straightforward and depend in part on the
assumptions of the researcher. For that reason, only observed or
"uncorrected" statistical relationships have been presented
in this preliminary analysis, although the "restriction of
the range" issue will be examined in subsequent research.