On the heels of an L.A. Times story that 31% of Americans are “still not angry yet at the nation’s media,” one wonders how much lower that percentage would drop if the public knew exactly how much they are not being told by the media.
Last week, Washington Post blogger Jay Mathews moved a provocative post about a research paper published in the Harvard Educational Review, purporting to provide ‘new evidence’ that the SAT is biased against African-American students and in favor of white students. On Monday, June 21, writer Scott Jaschik with the website Inside Higher Ed also moved a story on the report with similar structure and flow.
I wrote about this latest example of junk science last Sunday and have been thinking about what is behind all of this SAT bashing ever since. Standardized tests in general and the SAT in particular are two gigantic cultural piñatas. Because they are yardsticks by which achievement can be measured, they are despised by people who subscribe to the theory of equality of outcome rather than equality of opportunity.
If you take a look at what Mathews and Jaschik have done in their treatment of this report from the Harvard Educational Review – which, BTW, is not a peer reviewed journal – a scary trend begins to emerge. The style of reporting in both cases takes on an eerie and frightening tone that should give everyone reason to pause.
First we have the headlines. Here’s the hed for the Mathews piece:
And here’s the headline from the Jaschick piece:
New Evidence of Racial Bias on SAT
In both cases, the headlines contain an extremely serious accusation, one that’s akin to name calling. After all, if you or I were accused of hurting blacks or of racial bias, we’d take it pretty seriously and probably be indignant at the mere suggestion.
Now lets’ look at a few passages describing this research report. Here’s the fifth paragraph from the Mathews blog post:
“The confirmation of unfair test results throws into question the validity of the test and, consequently, all decisions based on its results,” said Maria Veronica Santelices, now at the Pontificia Universidad Catolica de Chile in Santiago, and Mark Wilson of UC Berkeley. “All admissions decisions based exclusively or predominantly on SAT performance–and therefore access to higher education institutions and subsequent job placement and professional success–appear to be biased against the African American minority group and could be exposed to legal challenge.”
Interestingly, we have this third paragraph from the Jaschik version:
The confirmation of unfair test results throws into question the validity of the test and, consequently, all decisions based on its results. All admissions decisions based exclusively or predominantly on SAT performance — and therefore access to higher education institutions and subsequent job placement and professional success — appear to be biased against the African American minority group and could be exposed to legal challenge,” says the study, which has just appeared in Harvard Educational Review.
Their use of identical quotes so high up in the respective stories – in both cases, it was the first quoted material – presents a broad and rather startling generality, one certain to grab attention.
While both writers did use quoted material from officials at the College Board, which runs the SAT, the treatment was largely dismissive, while test critics were given more favorable treatment. And in Jaschik’s case, facts given to him by the College Board that would negate claims of racial inequity were ignored, as I will explain in a minute.
Mathews noted the following:
[Researcher Saul] Geiser said he thinks the two researchers did a good job. He does not agree with ( College Board’s Laurence) Bunin’s criticisms of their work. He said he, like Freedle, wants more research on why blacks and whites answer these questions differently, so that any unfair disadvantages for blacks can be removed.
Jaschik didn’t bother with an academic for his story, choosing instead to use a quote from an anti-testing political group that goes by the name of Fair Test:
(Fair Test’s Bob) Schaeffer said that he agreed with the authors of the new study that use of the SAT could face legal challenges, given that this study now backs the finding that some of its questions may be harmful to the scores of black test-takers.
By using these quotes from these sources, Jaschik and Mathews are able to support the premise of the name-calling headline with other individuals who can transfer some other their authority to the subject.
We then have both writers trying to convey the notion that there is widespread belief that others believe the SAT is biased against African-American students. Here’s what Mathews wrote:
…the new paper means more researchers are likely to go more deeply into what Freedle has found…
And here’s how Jaschik treated it, again relying on a political group rather than an academic source:
More broadly, [Fair Test’s Bob Shaeffer] said that with more colleges considering ending SAT requirements, this new study is ‘another strong argument’ for doing so. ‘It’s going to add to the momentum.’
This is interesting because an enormous volume of research clearly shows the SAT treats all test takers the same and that the idea of the SAT being racial biased is “unequivocally rejected within mainstream psychology.” Because of this, Mathews and Jaschik have to find a way to further support their respective premises, so they turn to fringe sources rather than mainstream sources.
We’ve looked at a few things Mathews and Jaschik included in their reports but what about the things they did not include? In my story from June 20, I noted that the premise for the study’s conclusion of racial bias was based on two questions taken from four SAT tests used back during the Clinton Administration. These two questions showed that among kids of roughly the same academic performance, more white students got the correct answer and more African-Americans got a wrong answer, something researchers call differential item functioning or DIF.
Two questions from four tests show a significant DIF but does that merit calling the SAT racist? Let’s look at this another way. Those two questions with a significant DIF were also on one of the other SATs used in the study, but they were presented in a different order. Those exact same questions on a different test showed no significant DIF.
Neither Mathews nor Jaschik found these facts worthy of inclusion in their articles, which makes me wonder: Were they were deliberately ignoring them because calling the SAT racist will attract a lot more readers than a headline reading, “SAT is pretty much fair to everybody,” or did they just not read the study or understand what they were reading?
I know with absolute certainty that Jaschik was aware of these findings, as told to him by the College Board, because he told me so when I spoke to him on the phone requesting an interview from him on this topic. “That’s exactly what the college board told me,” he said.
My Mother’s Intuition tells me there something more at play here than just trying to sell newspapers or juice up their Internet traffic. Judging from these two reports on the study, it would appear that this idea of the SAT being racist is a movement sweeping across the nation. In truth it isn’t and in fact, both stories have many of the elements of classic propaganda techniques. Here are a few choice excerpts from the GMU treatise, which relies on the work of Edward Filene, who created the Institute for Propaganda Analysis:
Name Calling: Propandists use this technique to create fear and arouse prejudice by using negative words (bad names) to create an unfavorable opinion or hatred against a group, beliefs, ideas or institutions they would have us denounce. This method calls for a conclusion without examining the evidence.
Glittering Generalities: Propagandists employ vague, sweeping statements (often slogans or simple catchphrases) using language associated with values and beliefs deeply held by the audience without providing supporting information or reason.
Transfer: Transfer is a technique used to carry over the authority and approval of something we respect and revere to something the propagandist would have us accept.
Bandwagon: Propagandists use this technique to persuade the audience to follow the crowd. This device creates the impression of widespread support. It reinforces the human desire to be on the winning side. It also plays on feelings of loneliness and isolation.
Card Stacking: Propagandist uses this technique to make the best case possible for his side and the worst for the opposing viewpoint by carefully using only those facts that support his or her side of the argument while attempting to lead the audience into accepting the facts as a conclusion. In other words, the propagandist stacks the cards against the truth.
Taken as a whole these two writers have each managed to include five of a total of seven different propaganda techniques in their reporting on this issue. Four of the five we can see for ourselves but when it comes to the Card Stacking technique, it’s more difficult because it’s hard to really know what the author does and doesn’t know. But in this particular case, as I stated before, I know that Scott Jaschik was aware of the ‘two identical questions’ issue because he told me so himself last Friday, before his article had even been written (he told me he was going to be writing up his article soon when I spoke to him).
When day-to-day reporting on science or research-based issues more closely resembles the tried and true techniques of propaganda, we as a people and a nation have a lot to be genuinely afraid of. History has shown us the destructive power of propaganda and if these two articles on this issue are any indicator, we may be looking at the newest example of sacrificing the truth on the altar of radical ideology.
COMMENTS
Please let us know if you're having issues with commenting.