THE CHANGING WORLD OF COLLEGE ADMISSIONS TESTS (1)

Richard C. Atkinson
May 2002


It is obvious that our society is in the midst of rapid change, and that change is particularly evident here in California. An increasing proportion of our population consists of immigrants to California, and that trend holds particularly true among the younger age groups. The state's population today reflects incredible racial and cultural diversity. No other state - and no other country - has the range of races, ethnicities, languages, and cultures that characterize California today. At the same time, there are great differences among our state's high school students in their academic performance and in their preparation for a college education.

Given these realities, it behooves the University of California to ensure that our admissions processes are fair to all groups. But fairness is not enough - our admissions processes must also be perceived as fair.

With fairness and perceptions of fairness in mind, UC has made a series of significant changes in its undergraduate admissions process. I do not plan today to describe the changes in detail, but the initiatives - Eligibility in the Local Context, Dual Admissions, and Comprehensive Review - are explained fully on my web site, www.ucop.edu. Today, I intend to focus on the changes we have been discussing with respect to the standardized tests we use in admissions. For the sake of simplicity, my remarks will be framed around the SAT test. However, UC also accepts the ACT examination, and we are in discussions with the makers of both the SAT and the ACT about new examinations.

In order to talk about admissions tests for UC, I must first comment on the notion of eligibility for UC. We admit students from the top 12.5 percent of the statewide graduating high school class in California. How do we define the top 12.5 percent? There are two components. First, students must take the a-g courses in high school, a set of college preparatory courses in history, English, mathematics, laboratory science, foreign language, visual and performing arts, and college preparatory electives. Second, they must take the SAT I test of verbal and mathematics skills (or the ACT) and three SAT II achievement tests -writing, mathematics, and a third test of the student's choice selected from an array of options including literature, science, social studies, and foreign language. The formula for determining the top 12.5 percent is a combination of grades in the a-g courses and scores on the standardized tests. The university uses a sliding scale in which higher grades can offset lower test scores, and vice versa.

I have been a critic of the SAT I for many years - but, I would like to believe, an informed critic. I am not an expert in testing theory, but I am quite familiar with the testing literature and for several years served as chair of the National Academy of Sciences' Commission on Testing and Assessment. My unhappiness with the SAT was brought to a head, however, when I witnessed my 12-year-old granddaughter practicing verbal analogies as part of a series of exercises to prepare for the SAT - exercises aimed not at improving students' reading and writing abilities, but rather their test-taking skills.

Not long afterward, I was invited to give the opening address for the American Council on Education annual meeting in Washington, D.C., in February 2001. I left for Washington on a Friday, in advance of the Sunday speech, and checked into my hotel. On Saturday morning, I went downstairs to find a newspaper and was surprised to see stories about the speech on the front pages of The Washington Post and The New York Times, with significant sections of the speech reproduced verbatim. These stories marked the beginning of a period of national press attention to my proposals on standardized tests. Initially, I was chagrined at the leak of my speech text, being concerned that the audience on Sunday would feel short-changed. I soon discovered, however, that they would give me a very enthusiastic reception.

What was the substance of my speech? First, I did not call for the elimination of standardized tests. I believe strongly that we need standardized tests to provide an independent measure of student ability, given the distorting effects of grade inflation and differences in grading patterns among high schools. I did argue, however, that the tests should be demonstrably related to the college preparatory curriculum that students studied in high school. To that end, I proposed that UC move away from tests historically tied to the measurement of aptitude, such as the SAT I, and toward achievement tests that measure students' mastery of the college preparatory curriculum. Such a shift, I argued, would enhance the fairness of our testing requirements by clarifying our expectations of students seeking a UC education - that is, by strengthening the relationship between what students are taught in high school and what they are tested on for college admission.

I should say here that I have the highest regard for the College Board, which has responsibility for the development and administration of the SAT. They have remarkable technical expertise in the development of tests and a proven record in the security and administration of the tests. But I believe that a move toward achievement tests, and away from aptitude tests, would send a strong message to students that the best preparation for admissions tests is to take challenging courses in high school and excel in them - and that is the material they will be tested on for college admission. It also would send a clear message to high schools that mastery of the college preparatory curriculum is the focus of our interest, rather than mastery of test-taking skills.

The existing SAT I sends a confusing message to students and schools: that students will be tested on material not related to their course of study and that the grades they achieve can be devalued by tests unrelated to their school curriculum. Therefore, at the time of my speech last February, I also asked the UC faculty to examine our test requirements with the idea of making changes that would lead to the replacement of the SAT I.

Much has happened in the intervening period. Time Magazine ran a story on the issue, with a picture of President Bush and myself under the headline, "What Do These Two Men Have in Common?" One wag suggested the answer was our SAT scores - a rumor I must put to rest. As an applicant to the University of Chicago, I was not required to take the SAT. President Bush and I do, however, share an interest in improved educational achievement and the appropriate role of tests in fostering that achievement.

In addition, I have received hundreds of letters from educators, students, and parents, many with moving personal stories about their experiences with the SAT I. One in particular comes to mind - a cum laude graduate of UC Berkeley, with an MBA from Harvard, who said the first question she was asked when she went to interview for a job was what her SAT scores were. It is clear that the SAT has a deep impact on large numbers of Americans and that a discussion of its use and influence is long overdue.

Several months after I made my speech, two researchers at the UC Office of the President, Saul Geiser and Roger Studley, completed a seminal study on the predictive validity of college admissions tests. Because UC has required SAT I and SAT II scores of all its freshman applicants for many years, we have a unique data set. The Geiser-Studley paper used these data to examine the ability of high school grades and various combinations of standardized tests to predict student success in college. I will not discuss here the methodology of the study - the paper is available on my web site - but I want to mention the principal findings briefly.

First, the study found that high school grade-point average is the single-best predictor of freshman grades at UC. When a standardized test is used in combination with high school GPA, the SAT II is superior to the SAT I in predicting freshman grades. Adding the SAT I to the combination of high school GPA and SAT II yields virtually no improvement in the prediction. The SAT II also is less influenced by socioeconomic factors; its ability to predict freshman success varies by socioeconomic group much less than does the SAT I. Finally, next to high school GPA, the SAT II writing test is the best predictor of freshman success - a particularly important finding for me, because I have long advocated the importance of writing as a critical skill for college success. I made my testing proposal on educational policy grounds, not statistical grounds, but this analysis added much to the discussion of why we use the tests we do.

A UC faculty committee has been reviewing all these matters and has made a major contribution toward defining the type of admissions tests we should use in the University of California admissions process. Among other things, but perhaps most importantly, the committee's report called for the use of standardized tests with a much closer relationship to the college preparatory curriculum in high school. The faculty committee also worked closely with the College Board to discuss the possible development of a California test that would meet the committee's specifications.

In March of this year, however, I was pleased to receive a call from the president of the College Board, Gaston Caperton, saying his organization was considering a proposal to eliminate the SAT I as it now stands and replace it - on a national basis - with a new test very much in accord with the proposals that flowed from the UC faculty discussion. The College Board has been consulting with various groups about these proposed changes since March, and in June, the trustees of the organization are scheduled to vote on the changes. The new test, which would replace the existing SAT I for the entering college freshman class of 2006, would include several key elements: a writing section requiring students to produce an actual writing sample, a more substantial mathematics section assessing higher-level math skills, and a reading comprehension section that would not include verbal analogies.

I believe this is an ideal solution, reflecting a clear move toward achievement-based testing, and I am looking forward to the College Board's June meeting with great anticipation. I do hope, however, that the College Board also will consider renaming the test as a means of drawing a clear break with the SAT of days gone by. At UC, we would use this new test as a core test for admissions. In addition, we would continue to require students to take the SAT II subject examinations - perhaps one selected by the student in the area of humanities and literature, and another in the area of science and social science. The specific requirement will be a subject of discussion among the UC faculty in the coming months.

I greatly appreciate the attention that both the College Board and ACT have devoted to these issues. And I believe that the proposal to change the SAT I is in the best tradition of American higher education. The goal is to ensure that our universities and colleges are open to all young people - whatever their background - if they are willing to work hard in high school and master a college preparatory curriculum. These young people should be judged on the basis of their actual achievements in this regard and not on ill-defined notions of aptitude.

 1 Adapted from a speech delivered to the Western Association of College and University Business Officers, San Diego, California, May 7, 2002.