我國學生成就測驗中的猜測現象：1PL-AG之適用性及特點
Author(s):
Yi-Hung Lin (Department of Education, National Kaohsiung Normal University)
Abstract:
Guessing is a kind of random error and has impact on test reliability and validity. The guessing parameter of the 3PL (three-parameter logistic model) which proposed by A. Birnbaum in 1968, also termed pseudo-guessing, could be viewed as random guessing. The 3PL is currently the main reference model of related studies. The 1PL-AG (one parameter logistic model with ability-based guessing) which proposed by San Martín et al. in 2006, suggests that guessing is related to ability, i.e., a high-ability examinee tends to have more partial knowledge that may lead to a higher success rate if guess. Since the 1PL-AG is based on the analysis results of Chilean student achievement tests, considering students in Taiwan and Chile have different learning achievements and cultures and may have different guessing behaviors, this study compares model-data fit indexes of 1PLAG and another competing model for two empirical datasets to examine the applicability of 1PL-AG. In order to supplement the uninvestigated issues in San Martín et al.’s study, the obtained parameter estimates from empirical analyses are used to generate simulated datasets to explore the influences of ignoring the ability-weighted parameter α. The results of empirical data analyses are consistent with San Martín et al.’s study that 1PL-AG fits two datasets both significantly better than the competing model, but the values of parameter α are larger in Taiwan’s datasets, which means that students in Taiwan tend to guess with their ability, compared to students in Chile. The consequences of simulation analyses reveal that larger estimation biases for ability and item parameters may be obtained if the ability-weighted parameter α is ignored, and the magnitude of bias is influenced by the size of parameter α: The larger the parameter α being ignored, the larger the estimation bias is. Additionally, ignoring extreme parameter α (e.g. 3.45 in the current study) may lead to more extreme ability estimates, lower item difficulty estimates, and more centered guessing parameter estimates; ignoring non-extreme parameter α (e.g. 0.46 in the current study) may lead to more centered ability estimates, item difficulty and guessing parameter estimates. Suggestions are provided according to these results.
Keywords:
1PL-AG、ability-based、guessing、partial knowledge
