论文部分内容阅读
本研究采用混合研究法分析了评分经验对评分人差异的影响。27位CET–4作文评分人按照评分经验平均分为老手、中手和新手三组。通过严格培训后,他们对30篇CET–4模拟作文评分,并按重要性依次排列提供三条评分理由。分数的多层面Rasch模型(MFRM)结果表明:1)三组在严厉度上不存在显著差异;2)老手评分内在一致性最好,新手次之,中手最差;3)除一位新手外,三组都没有出现明显的随机效应。评分标准相关理由编码百分比的混合多元方差分析(MANOVA)结果表明:评分经验的主效应、评分经验和评分理由重要性的交互效应都不显著,但是评分理由重要性的主效应显著。这一结果说明经过培训后,评分经验产生的评分差异可以得到有效控制。
In this study, a mixed study was used to analyze the influence of scoring experience on the difference between scorers. 27 CET-4 essayers were divided into three groups: veteran, midfielder and novice. After rigorous training, they rated 30 CET-4 mock essays and ranked them three times in order of importance. The multi-level Rasch model (MFRM) scores of scores showed that: 1) there was no significant difference between the three groups in severity; 2) the veteran’s score was the most consistent among the novice, followed by the worst among the hands; 3) However, no obvious random effects were observed in all three groups. MANOVA results showed that the main effect of scoring experience, the experience of scoring and the reciprocity effect of the importance of scoring reason were not significant, but the main effect of scoring rationality was significant. This result shows that after training, the difference in ratings produced by the scoring experience can be effectively controlled.