Differential performance, standardized testing and race

Writing Differences in Teacher Performance Assessments: An Investigation of African American Language and Edited American English


This study sought to identify the source of racial disparity in test scores for the National Board for Professional Teaching Standards (NBPTS) portfolio assignment for the Middle Childhood/Generalist Certificate. The authors of the study begin with the hypothesis that the use of features African American Language (AAL) and Southeastern White English (SWE) may contribute to lower scores on this test. The researchers examined thirty-two written portfolio entries with 18 from African American candidates and 14 from European American candidates. These entries were coded by linguistic experts for grammatical, lexical, and discourse features and most notably for features of AAL, SWE and Speech Code Errors (SCE). The race of the writer was kept confidential to the coders.

The results revealed that the use of AAL features was found among African American candidates across all score levels and that African American candidates used both AAL and SCE more often than the European American candidates. Interestingly though, the study found that the use of AAL and SCE was not associated with high or low scores, lending support to the effectiveness of the testing body’s bias-reduction training. The researchers conclude that although this study did not reveal a bias toward non-standard English users, African American testers still received lower test scores than European Americans. Of the participants in this research, there was an approximately half-point difference in the mean scores between African American and European American participants; a finding that is consistent with results from previous participant cohorts of the same test. The researchers hypothesize that “it is possible that some of this differential performance could be due to construct-irrelevant effect of the writing features used (M.Y. Szpara & Wylie, 2007)

Although the study was inconclusive regarding whether linguistic bias against non-standard English use in standardized tests, I found this study to be a useful beginning for discourse regarding race, writing and differential performance. For my research, I find this study to be interesting because it shows that linguistic experts are very adept at identifying race in writing but questions whether the common person can do so as easily. It also questions whether having the ability to distinguish racial identity through writing creates an inherent bias against writers who are identified as a minority. As I will asking the participants of my study to attempt to identify race, age and gender through casual writing, I find this study to be a great point of comparison. As the expert linguistics in this study coded AAL and SWE features, I will be asking non-experts to identify codes that may denote the writer’s demographic categories. The comparison between the ability of experts to identify demographic information through writing and that of non-experts, I believe will determine new areas of discourse in the subjects of digital writing, race and linguistics.

Szpara, M. Y., & Wylie, E. C. (2007). Writing differences in teacher performance assessments: An investigation of African American language and edited American English. Applied Linguistics, 29(2), 244-266.