References
AlKhuzaey, Samah, Floriana Grasso, Terry R. Payne, and Valentina Tamma. 2024. “Text-Based Question Difficulty Prediction: A Systematic Review of Automatic Approaches.” International Journal of Artificial Intelligence in Education 34 (3): 862–914. https://doi.org/10.1007/s40593-023-00362-1.
American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. 2014. Standards for Educational and Psychological Testing. American Educational Research Association.
Argyle, Lisa P., Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023. “Out of One, Many: Using Language Models to Simulate Human Samples.” Political Analysis 31 (3): 337–51. https://doi.org/10.1017/pan.2023.2.
Ashton, Michael C., Kibeom Lee, and Lewis R. Goldberg. 2007. “The IPIP–HEXACO Scales: An Alternative, Public-Domain Measure of the Personality Constructs in the HEXACO Model.” Personality and Individual Differences 42 (8): 1515–26. https://doi.org/10.1016/j.paid.2006.10.027.
Ayala, R. J. de. 2022. The Theory and Practice of Item Response Theory. 2nd ed. Guilford Press.
Badrinath, Anirudhan, Frederic Wang, and Zachary A. Pardos. 2021. “pyBKT: An Accessible Python Library of Bayesian Knowledge Tracing Models.” Proceedings of the 14th International Conference on Educational Data Mining, 468–74.
Benedetto, Luca, Paolo Cremonesi, Andrew Caines, et al. 2023. “A Survey on Recent Approaches to Question Difficulty Estimation from Text.” ACM Computing Surveys 55 (9): Article 178. https://doi.org/10.1145/3556538.
Bock, R. Darrell, and Murray Aitkin. 1981. “Marginal Maximum Likelihood Estimation of Item Parameters: Application of an EM Algorithm.” Psychometrika 46 (4): 443–59.
Breiman, Leo. 2001. “Statistical Modeling: The Two Cultures.” Statistical Science 16 (3): 199–231. https://doi.org/10.1214/ss/1009213726.
Bridgeman, Brent, Catherine Trapani, and Yigal Attali. 2012. “Comparison of Human and Machine Scoring of Essays: Differences by Gender, Ethnicity, and Country.” Applied Measurement in Education 25 (1): 27–40. https://doi.org/10.1080/08957347.2012.635502.
Bulut, Okan, Maggie Beiting-Parrish, Jodi M. Casabianca, et al. 2024. “The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges.” Chinese/English Journal of Educational Measurement and Evaluation, ahead of print. https://doi.org/10.59863/MIQL7785.
Bulut, Okan, Guher Gorgun, and Seyma Nur Yildirim-Erbasli. 2021. “Estimating Explanatory Extensions of Dichotomous and Polytomous Rasch Models: The Eirm Package in R.” Psych 3 (3): 308–21. https://doi.org/10.3390/psych3030023.
Bulut, Okan, Jinnie Shin, Seyma N. Yildirim-Erbasli, Guher Gorgun, and Zachary A. Pardos. 2023. “An Introduction to Bayesian Knowledge Tracing with pyBKT.” Psych 5 (3): 770–86. https://doi.org/10.3390/psych5030050.
Chalmers, R. Philip. 2012. “mirt: A Multidimensional Item Response Theory Package for the R Environment.” Journal of Statistical Software 48 (6): 1–29. https://doi.org/10.18637/jss.v048.i06.
Cizek, Gregory J., and Michael B. Bunch. 2007. Standard Setting: A Guide to Establishing and Evaluating Performance Standards on Tests. Sage.
Cohen, Jacob. 1968. “Weighted Kappa: Nominal Scale Agreement with Provision for Scaled Disagreement or Partial Credit.” Psychological Bulletin 70 (4): 213–20.
Corbett, Albert T., and John R. Anderson. 1995. “Knowledge Tracing: Modeling the Acquisition of Procedural Knowledge.” User Modeling and User-Adapted Interaction 4 (4): 253–78. https://doi.org/10.1007/BF01099821.
Crocker, Linda, and James Algina. 1986. Introduction to Classical and Modern Test Theory. Holt, Rinehart; Winston.
De Boeck, Paul, and Mark Wilson, eds. 2004. Explanatory Item Response Models: A Generalized Linear and Nonlinear Approach. Springer. https://doi.org/10.1007/978-1-4757-3990-9.
DeCarlo, Lawrence T. 2011. “On the Analysis of Fraction Subtraction Data: The DINA Model, Classification, Latent Class Sizes, and the Q-Matrix.” Applied Psychological Measurement 35 (1): 8–26.
Deonovic, Benjamin, Michael Yudelson, Maria Bolsinova, Meirav Attali, and Gunter Maris. 2018. “Learning Meets Assessment: On the Relation Between Item Response Theory and Bayesian Knowledge Tracing.” Behaviormetrika 45 (2): 457–74. https://doi.org/10.1007/s41237-018-0070-z.
Doignon, Jean-Paul, and Jean-Claude Falmagne. 1999. Knowledge Spaces. Springer.
Drasgow, Fritz, Michael V. Levine, and Esther A. Williams. 1985. “Appropriateness Measurement with Polychotomous Item Response Models and Standardized Indices.” British Journal of Mathematical and Statistical Psychology 38 (1): 67–86.
Embretson, Susan E., and Steven P. Reise. 2000. Item Response Theory for Psychologists. Lawrence Erlbaum Associates.
Fischer, Gerhard H. 1973. “The Linear Logistic Test Model as an Instrument in Educational Research.” Acta Psychologica 37 (6): 359–74. https://doi.org/10.1016/0001-6918(73)90003-6.
Geerlings, Hanneke, Cees A. W. Glas, and Wim J. van der Linden. 2011. “Modeling Rule-Based Item Generation.” Psychometrika 76 (2): 337–59. https://doi.org/10.1007/s11336-011-9204-x.
Gierl, Mark J., and Thomas M. Haladyna, eds. 2013. Automatic Item Generation: Theory and Practice. Routledge.
Gierl, Mark J., and Hollis Lai. 2013. “Instructional Topics in Educational Measurement (ITEMS) Module: Using Automated Processes to Generate Test Items.” Educational Measurement: Issues and Practice 32 (3): 36–50. https://doi.org/10.1111/emip.12018.
Glas, Cees A. W., and Wim J. van der Linden. 2003. “Computerized Adaptive Testing with Item Cloning.” Applied Psychological Measurement 27 (4): 247–61. https://doi.org/10.1177/0146621603027004001.
Hambleton, Ronald K., H. Swaminathan, and H. Jane Rogers. 1991. Fundamentals of Item Response Theory. Sage.
Hao, Jiangang, Alina A. von Davier, Victoria Yaneva, Susan Lottridge, Matthias von Davier, and Deborah J. Harris. 2024. “Transforming Assessment: The Impacts and Implications of Large Language Models and Generative AI.” Educational Measurement: Issues and Practice 43 (2): 16–29. https://doi.org/10.1111/emip.12602.
Holland, Paul W., and Dorothy T. Thayer. 1988. “Differential Item Performance and the Mantel-Haenszel Procedure.” In Test Validity, edited by Howard Wainer and Henry I. Braun. Lawrence Erlbaum Associates.
Holland, Paul W., and Howard Wainer, eds. 1993. Differential Item Functioning. Lawrence Erlbaum Associates.
International Test Commission, and Association of Test Publishers. 2022. Guidelines for Technology-Based Assessment.
James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2021. An Introduction to Statistical Learning with Applications in R. 2nd ed. Springer. https://doi.org/10.1007/978-1-0716-1418-1.
Johnson, Matthew S., Xiang Liu, and Daniel F. McCaffrey. 2022. “Psychometric Methods to Evaluate Measurement and Algorithmic Bias in Automated Scoring.” Journal of Educational Measurement 59 (3): 338–61. https://doi.org/10.1111/jedm.12335.
Kane, Michael T. 2013. “Validating the Interpretations and Uses of Test Scores.” Journal of Educational Measurement 50 (1): 1–73. https://doi.org/10.1111/jedm.12000.
Kolen, Michael J., and Robert L. Brennan. 2014. Test Equating, Scaling, and Linking: Methods and Practices. 3rd ed. Springer.
Leighton, Jacqueline P., Mark J. Gierl, and Stephen M. Hunka. 2004. “The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka’s Rule-Space Approach.” Journal of Educational Measurement 41 (3): 205–37.
Linacre, John M. 1989. Many-Facet Rasch Measurement. MESA Press.
Linden, Wim J. van der. 2005. Linear Models for Optimal Test Design. Springer.
Lord, Frederic M. 1980. Applications of Item Response Theory to Practical Testing Problems. Lawrence Erlbaum Associates.
Loukina, Anastassia, Nitin Madnani, Aoife Cahill, et al. 2020. “Using PRMSE to Evaluate Automated Scoring Systems in the Presence of Label Noise.” Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, 18–29. https://aclanthology.org/2020.bea-1.2.
Masters, Geoff N. 1982. “A Rasch Model for Partial Credit Scoring.” Psychometrika 47 (2): 149–74.
Maydeu-Olivares, Alberto, and Harry Joe. 2006. “Limited Information Goodness-of-Fit Testing in Multidimensional Contingency Tables.” Psychometrika 71 (4): 713–32.
Muraki, Eiji. 1992. “A Generalized Partial Credit Model: Application of an EM Algorithm.” Applied Psychological Measurement 16 (2): 159–76.
Orlando, Maria, and David Thissen. 2000. “Likelihood-Based Item-Fit Indices for Dichotomous Item Response Theory Models.” Applied Psychological Measurement 24 (1): 50–64.
Pardos, Zachary A., and Neil T. Heffernan. 2011. “KT-IDEM: Introducing Item Difficulty to the Knowledge Tracing Model.” User Modeling, Adaption and Personalization, 243–54. https://doi.org/10.1007/978-3-642-22362-4_21.
Piech, Chris, Jonathan Bassen, Jonathan Huang, et al. 2015. “Deep Knowledge Tracing.” Advances in Neural Information Processing Systems 28.
Rudner, Lawrence M. 2001. “Computing the Expected Proportions of Misclassified Examinees.” Practical Assessment, Research, and Evaluation 7 (14).
Samejima, Fumiko. 1969. “Estimation of Latent Ability Using a Response Pattern of Graded Scores.” Psychometrika Monograph Supplement 34 (4, Pt. 2).
Sinharay, Sandip, Matthew S. Johnson, and David M. Williamson. 2003. “Calibrating Item Families and Summarizing the Results Using Family Expected Response Functions.” Journal of Educational and Behavioral Statistics 28 (4): 295–313.
Stamper, John, Alexandru Niculescu-Mizil, Steven Ritter, Geoffrey J. Gordon, and Kenneth R. Koedinger. 2010. Challenge Data Sets from KDD Cup 2010. PSLC DataShop. https://pslcdatashop.web.cmu.edu/KDDCup/.
Stocking, Martha L., and Frederic M. Lord. 1983. “Developing a Common Metric in Item Response Theory.” Applied Psychological Measurement 7 (2): 201–10.
Tatsuoka, Curtis. 2002. “Data Analytic Methods for Latent Partially Ordered Classification Models.” Journal of the Royal Statistical Society: Series C (Applied Statistics) 51 (3): 337–50.
Tatsuoka, Kikumi K. 1984. Analysis of Errors in Fraction Addition and Subtraction Problems. Final Report NIE-G-81-0002. University of Illinois, Computer-based Education Research Laboratory.
Templin, Jonathan, and Laine Bradshaw. 2014. “Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies.” Psychometrika 79 (2): 317–39.
Torre, Jimmy de la, and Jeffrey A. Douglas. 2004. “Higher-Order Latent Trait Models for Cognitive Diagnosis.” Psychometrika 69 (3): 333–53.
Wainer, Howard et al. 2000. Computerized Adaptive Testing: A Primer. 2nd ed. Lawrence Erlbaum Associates.
Warm, Thomas A. 1989. “Weighted Likelihood Estimation of Ability in Item Response Theory.” Psychometrika 54 (3): 427–50.
Williamson, David M., Xiaoming Xi, and F. Jay Breyer. 2012. “A Framework for Evaluation and Use of Automated Scoring.” Educational Measurement: Issues and Practice 31 (1): 2–13. https://doi.org/10.1111/j.1745-3992.2011.00223.x.
Yan, Duanli, André A. Rupp, and Peter W. Foltz, eds. 2020. Handbook of Automated Scoring: Theory into Practice. CRC Press.
Yaneva, Victoria, Kai North, Peter Baldwin, et al. 2024. “Findings from the First Shared Task on Automated Prediction of Difficulty and Response Time for Multiple-Choice Questions.” Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications, 470–82. https://aclanthology.org/2024.bea-1.39.
Yarkoni, Tal, and Jacob Westfall. 2017. “Choosing Prediction over Explanation in Psychology: Lessons from Machine Learning.” Perspectives on Psychological Science 12 (6): 1100–1122. https://doi.org/10.1177/1745691617693393.
Yen, Wendy M. 1984. “Effects of Local Item Dependence on the Fit and Equating Performance of the Three-Parameter Logistic Model.” Applied Psychological Measurement 8 (2): 125–45.