Use of differential item functioning (DIF) analysis for bias analysis in test construction

Abstract

When differential item functioning (DIF) item analysis procedures based on item response theory (IRT) are used during test construction, it is possible to draw item characteristic curves for the same item for different subgroups. These curves indicate how each item functions at various ability levels for different subgroups. DIF is indicated by the area between the curves. In the construction of the Learning Potential Computerised Adaptive Test (LPCAT), this method was used to identify items that indicated bias in terms of gender, culture, language or level of education. Items that exceeded a predetermined amount of DIF were discarded from the final item bank, irrespective of which subgroup was being advantaged or disadvantaged. The process and results of the DIF analysis are discussed.