Russell T. Warne
Standardized tests are one of those topics that many people have an opinion about, despite most people being uninformed. Memories of filling in bubble sheets during childhood or anxiety about college admissions tests color people’s perceptions. Additionally, the highly technical field of test development and the confidentiality surrounding test content (to prevent cheating) makes standardized tests seem like a “black box.” As a result, many incorrect beliefs have sprung up about these tests.
One of the most common incorrect beliefs about tests is that they are biased against racial and ethnic minorities. It is easy to see where this belief comes from. In many countries, marginalized and underrepresented groups have lower average scores than the averages found in politically, socially, and economically dominant groups. Despite the tremendous amount of overlap in scores across groups, the differences between averages is often a noticeable characteristic of score reports. A reasonable explanation for many members of the media and the public is that something must be wrong with the tests and that they must be bias towards minorities.
The reality is more complex than this. Differences in average scores can occur for many reasons, one of which is bias in the test. Psychologists have studied test bias for over 50 years, and today they have many tools for investigating test bias.
One important result of the research on test bias is a fundamental understanding of what, exactly, test bias is. Whereas laymen understand average score differences as being evidence of test bias, the technical definition among experts is more complex. For the testing industry, bias occurs when two people from different demographic groups—who are equal in ability—earn different scores on a test because of their demographic group membership. Thus, identifying test bias requires looking for differences that emerge in these groups of equal ability, who speak the same native language, are acculturated to the country, etc.
This is not to say that average score differences are irrelevant. Indeed, they are often a good indicator that the psychologists should investigate whether test bias is present. But—by themselves—they are not sufficient to show that a test is actually biased.
So how common is test bias in professionally developed tests? The answer is that it is almost non-existent. Today, it is routine for professionally developed tests to be screened for bias before they are sold or administered on a large scale, and doing so is part of the field’s ethics code and professional standards. Moreover, the underlying cognitive architecture of intelligence and related abilities seems remarkably stable across cultures, thus making it easier to create a test that can function for multiple groups.
Unfortunately, what psychologists have known for decades about test bias has not filtered to the general public. As a result, tests often come under attack because well-meaning outsiders see them as an engine of unfairness. However, an unbiased test can be a tool to strengthen society because it can be a barometer of progress and identify individual excellence in people from all backgrounds.
Enjoyed reading this article? Share it today:
