2007 UPDATE (August): When this essay originally appeared, it inspired a column at the San Antonio News-Express. The column led to an interview with a San Antonio radio station. The Educational Testing Service has since released three new GRE data sets. See Endnote #1 for side-by-side comparisons of these later scores with the scores in this essay. As you will see, mean GRE scores are remarkably stable. We are looking at a persistent phenomenon.

Mean Verbal - 429, Mean Quantitative - 520, Total - 949

Those are the mean Graduate Record Examination (GRE) scores of applicants for graduate study in Education Administration tested between July 1, 2000 and June 30, 2003. Of 51 intended areas of graduate study, applicants in 45 fields had higher Total GRE scores than applicants in Education Administration. Candidates in 5 fields — Home Economics, Social Work, Student Counseling, Early Childhood and Special Education — had lower total GRE scores.

Verbal and Quantitative scores follow the same pattern. Applicants in 46 fields had higher Verbal scores than candidates in Education Administration. Applicants in 4 fields had lower Verbal scores. (The data seem to explain why books, articles and Op-Eds by education administrators are rarities.)

Applicants in 45 fields had higher Quantitative scores that aspiring Education Administrators. Applicants in 5 fields had lower Quantitative scores.

For context, below are mean GRE scores (Verbal, Quantitative, Total) of applicants in broad traditional fields and of all 1,206,000 examinees tested:

Engineering - 468, 721, 1189

Physical Sciences - 488, 699, 1187

Humanities/Arts - 541, 561, 1102

Life Sciences - 464, 580, 1044

Social Sciences - 485, 559, 1044

Business - 448, 591, 1039

Education - 450, 531, 981

Mean for All Examinees - 470, 598, 1068

Each traditional field has subcategories. The full data set is available here (PDF file). Scroll down to pages 18-20. (1)

What do the data on Graduate Record Examination scores of aspiring Education Administrators mean? What do the data tell us about the quality of decision-making in state education departments, in district central offices and in principals' offices? How are curricula, textbooks, instructional programs, teachers impacted by school leaders with such academic aptitude scores? To answer that question some background on academic aptitude tests — the SAT, the Scholastic Aptitude Test, and the GRE — is in order. Both tests are produced by the Educational Testing Service (ETS).

Testing the Tests

The Big Test: The Secret History of the American Meritocracy (1999) by Nicholas Lemann, Dean of the Columbia School of Journalism, is largely about the SAT: its origins (in the Army IQ tests used in World War I), its rise to dominance in college admissions, and its cultural impact. The Big Test details the Educational Testing Service's response to challenges over the years: from rivals, from test critics, from test prep companies and from heightened social concerns about diversity. The Big Test is a beautifully-written, 350-page history of the nation's academic sorting process. I found it enthralling, notwithstanding its flawed prescription: a national high school curriculum. (2)

The SAT's purpose is to more fairly assess the academic aptitude of students attending high schools varying in rigor and quality. The GRE has a similar purpose: to fairly assess the academic aptitude of students who are graduating/have graduated from a wide range of colleges. Roughly 430,000 grad school hopefuls take the GRE each year.

As I wrote in "Standardized Tests That Fire the Imagination," the verbal parts of both the SAT (and the GRE) are flawed. They are culturally sterile. This cultural sterility has a pernicious effect on curricula at every level. Culture-free high-stakes standardized reading tests — which include the SAT and GRE — drive a content-free K-12 curriculum. They produce — over many years— ignorant young people. This is why high school and college graduates know so little history and literature. Young Americans are culturally illiterate.

Nonetheless, Verbal SAT and GRE scores have meaning. Not surprisingly, of the 51 fields for which ETS tabulated GRE scores, candidates in Philosophy and in English, Language and Literature had the highest mean Verbal scores, 588 and 557, respectively.

The Quantitative sections of both tests — the "Math" tests, as they are known informally — are of high quality.

The Quantitative tests seem to have been an unalloyed boon for America. Steve Sailer, founder of the Human Biodiversity Institute:

The Math SAT "liberated a group so dispersed and downtrodden that it didn't even have a name until about 30 years ago: nerds. By identifying nerdy geniuses in high schools across America and Canada, many of whom were too bored to make good grades, the Math SAT enabled them to form critical masses of computer geeks in nerd havens like Stanford and MIT. Out of these colleges grew the great high-tech incubators such as Silicon Valley and Route 128 (around Boston), which are the engines of the current American boom." (Canadian National Post , 10/99)

Bravo for the Math SAT. Having struggled through an engineering program at Duke University with its fair share of "nerdy geniuses," by my standards. I am also an admirer of the Math SAT and the Math GRE.

GRE and SAT scores have meaning.

This is not to ignore these tests' great limitations. Lemann:

"Merit is various, not unidimensional. Intelligence tests, and also education itself, can't be counted on to find every form of merit. They don't find wisdom, or originality, or humor, or toughness, or empathy, or common sense, or independence, or determination — let alone moral worth."

Lemann's point deserves emphasis. A GRE/SAT score does NOT get at originality, humor, toughness, empathy, common sense, independence, determination. Who has not known a high-scoring dud severely lacking in one or more such qualities?

Moreover, some large part of "academic aptitude" is earned by sheer doggedness in verbal or mathematical endeavors. These need not cease at age 18 or 22. And the tests are coachable to some degree. (3)

But for all their limitations, the GRE and the SAT measure what they purport to measure with some success. There is, therefore, a special irony if — as seems certain from the data — we are selecting as heads of our K-12 academic institutions, individuals who demonstrate negligible academic promise.

Cash Cows for Universities

To be fair, we do not know what percentage of applicants to Education Administration programs are accepted. We can, however, make a well-supported guess: close to 100 percent. "Educating School Leaders," by Arthur Levine, president, Columbia Teachers College, is a study of Education Leadership programs nationwide. Excerpt from a New York Times report on the Levine study:

All states, and nearly all public school districts within them, award higher salaries to teachers who take additional courses and earn advanced degrees. One result of this has been an 'army of unmotivated' educators looking for extra credits 'in the easiest ways possible' during their off hours, the report said. The universities, in turn, capitalize on this demand by viewing their education schools as 'cash cows,' setting low admissions standards and offering 'quickie degrees' instead of investing in a quality curriculum . . . (March, 2005)

It is a decent bet that few applicants to Education Administration programs are rejected.

We still do not know what percentage of students in Educational Administration programs are or will become school administrators. Some may be teachers seeking an Education Administration Credential and/or a salary boost from the easy credits, which would not much brighten the picture.

Curiously, the mean Total GRE score of applicants for graduate study in Education — broadly-speaking — is 981. That of aspiring Education Administrators: 949. That might make for some interesting differences of opinion on curricular issues.

"Among the Lowest in All Academe"

Academic achievement is not the only route to success in a free-market society such as the United States. One can step off the schooling treadmill (how I saw it) and carve one's own path to success. But if we are going to establish institutions — public schools — whose mission is academic: the transmission of academic skills and knowledge, it would be wise to put only those with strong academic credentials in charge.

We do the opposite.

We select leaders for our school systems from a pool of individuals whose standardized test scores "are not only among the lowest in education-related fields but are among the lowest in all academe." (4)

This is lunacy.

Endnotes

1) 2007 UPDATE (8-07): Data are now available from 2005-2006 GRE Guide to the Use of Scores, which applies to the 1,250,000 applicants tested between July 1, 2001, and June 30, 2004. Scroll to pages 18-20. Data are available (but temporarily off-line) on the 2006-2007 GRE Guide to the Use of Scores, which applies to the 1,245,000 applicants tested between July 1, 2002 and June 30, 2005. Data have just become available from the 2007-2008 GRE Guide, which applies to the 1,223,000 applicants tested between July 1, 2003 and June 30, 2006. Highlights:

Mean scores for all fields of study exhibit minimal fluctuation from year to year. Example: For applicants for graduate work in Education Administration, here are Verbal, Quantitative and Mean Total GRE scores found in the 2004-2005 GRE Guide (These data appear in the body of the essay), in the 2005-2006 GRE Guide, in the 2006-2007 GRE Guide, and in the 2007-2008 GRE Guide, respectively::

Verbal: 429, 427, 426, 426; Quantitative: 520, 523, 522, 520; Total: 949, 950, 948, 946

Different time periods, different examinees, nearly identical scores. Let's see if this holds true for mean GRE scores in broad traditional fields across those same time periods:

Engineering (Chemical, Civil, Electrical, Industrial, Materials, Mechanical)

Verbal: 468, 467, 468, 470; Quantitative: 721, 720,719, 718; Total: 1189, 1187, 1187, 1188

Physical Sciences (Chemistry, Computer, Earth, Mathematical, Physics)

Verbal: 488, 487, 486, 486; Quantitative: 699, 699, 697, 694; Total: 1187,1186, 1183, 1180

Humanities/Arts (Art History, Performance, English Lit, Foreign Lit, History, Philosophy)

Verbal: 541, 544, 545, 545; Quantitative: 561, 566, 566, 564; Total: 1102, 1110, 1111, 1109

Life Sciences (Agriculture, Biological Sciences, Health Sciences)

Verbal: 464, 463, 462, 460; Quantitative: 580, 582, 581, 578; Total: 1044, 1045, 1043, 1038

Social Sciences (Anthropology, Economics, Political Science, Psychology, Sociology)

Verbal: 485, 486, 487, 487; Quantitative: 559, 565, 565, 563; Total: 1044, 1051, 1052, 1050

Business (Accounting, Banking & Finance, Administration & Management)

448, 446, 442, 440; Quantitative: 591, 595, 592, 591; Total: 1039, 1041, 1034, 1031

Education (Administration, Curriculum & Instruction, Early Childhood, Elementary, Evaluation & Research, Higher, Secondary, Special, Student Counseling & Personnel Services)

Verbal: 450, 450, 449, 449; Quantitative: 531, 534, 534, 533; Total: 981, 984, 983, 982

Mean for All Examinees (approx. 1.2 million for each three-year period)

Verbal: 470, 469, 467, 465; Quantitative: 598, 597, 591, 594; Total: 1068, 1066, 1058, 1059

2) Maggie Campbell Gallagher, a New York free-lance writer, Harvard-trained lawyer and owner of a bar-exam preparation business, has written a superb review of The Big Test .

3) The coaching effect is not nearly as great as test-prep companies would have the public believe according to Charles Murray, author of The Bell Curve, a 1994 best-seller:

"From 1981 to 1990, three separate analyses of all the prior studies [on effects of coaching on SAT performance] were published in peer-reviewed journals. They found a coaching effect of 9 to 25 points on the SAT Verbal and 15 to 25 points on the SAT Math. In 2004, Derek Briggs, using the National Education Longitudinal Study of 1988, found effects of 3 to 20 points for the SAT Verbal and 10 to 28 points for the SAT Math. Donald powers and Donald Rock, using a nationally representative sample of students who took the SAT after its revisions in the mid-1990s, found an average coaching effect of 6 to 12 points on the SAT Verbal and 13 to 18 points on the SAT Math. Many studies tell nearly identical stories. On average, coaching raises scores by no more than a few dozen points, enough to sway college admissions in exceedingly few cases."

"I am not reporting a scholarly literature with a two-sided debate. No study published in a peer-reviewed journal shows average gains approaching the fabled 100-point and 200-point jumps you hear about in anecdotes. While preparing this article, I asked Kaplan and Princeton Review [test prep companies] for such evidence. Kaplan replied that it chooses not to release data for proprietary reasons. Princeton Review did not respond at all." ("Abolish the SAT," by Charles Murray, The American, July/August, 2007)

4) "A Race to the Bottom: The Nation's School Leadership Programs are not Producing the Educational Administrators We Need," by Arthur Levine, National Crosstalk , Vol. 13, No. 3 - Summer, 2005.

Tom Shuford [email protected] is a retired teacher living in Lenoir, North Carolina.

Published July 27, 2006