Fits an explanatory item response model in which the knowledge graph predicts how hard each item is: A linear logistic test model (LLTM; Fischer, 1973) with a random item effect for the part of difficulty the graph leaves unexplained (De Boeck, 2008), estimated with the eirm package (Bulut et al., 2021). Two sets of predictors are offered:
Usage
mv_kg_explain(kg, data, predictors = c("concepts", "depth"), covariates = NULL)Arguments
- kg
a knowledge graph from
mv_kg_build()oras_mv_kg().- data
a data frame or matrix of 0/1 item scores; columns that are not items of the graph, such as an ID, are skipped.
- predictors
"concepts"or"depth".- covariates
optional names of item metadata columns to add as predictors (see
mv_kg_annotate()).
Value
An object of class mv_kg_explain: A list with effects (one
row per predictor, with its difficulty, standard error and p
value), items (each item's difficulty in the Rasch model, the
difficulty the graph predicts, and the residual), fit (the two
models' variances and information criteria), and the two
eirm::eirm() fits in model and base. Difficulties are on the
logit scale, with higher values for harder items.
Details
"concepts": One indicator per concept, from the graph's Q-matrix. Each concept gets its own difficulty, and an item that needs several concepts gets the sum."depth": The item's place in the learning order (the largest prerequisite depth among its concepts) and the number of concepts it needs. This tests the graph's claim that later concepts are harder.
Item metadata added with mv_kg_annotate(), such as word count or item
format, can join either set through covariates. Numeric columns are
centred, so the concept difficulties refer to an item of average value;
other columns get one 0/1 indicator for each value but the first. This
asks whether a feature of the items explains difficulty beyond what the
graph does.
A Rasch model with random items and no predictors is fitted too, and
explained in glance() is the share of its item variance that the
predictors account for.
References
Bulut, O., Gorgun, G., & Yildirim-Erbasli, S. N. (2021). Estimating explanatory extensions of dichotomous and polytomous Rasch models: The eirm package in R. Psych, 3(3), 308-321.
De Boeck, P. (2008). Random item IRT models. Psychometrika, 73(4), 533-559.
Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37(6), 359-374.
See also
predict_difficulty() for difficulty from item text features.
Examples
# \donttest{
if (requireNamespace("eirm", quietly = TRUE)) {
ex <- mv_kg_explain(fractions_kg, fractions)
ex
tidy(ex)
mv_kg_explain(fractions_kg, fractions, predictors = "depth")
# does the item format explain difficulty beyond the concepts?
kg <- mv_kg_annotate(fractions_kg, data.frame(
item = sprintf("F%02d", 1:24), format = rep(c("choice", "open"), 12)))
mv_kg_explain(kg, fractions, covariates = "format")
}
#> <mv_kg_explain> Grade 5 fractions: Item difficulty explained by concepts and item metadata
#> 24 items, N = 1200. The graph explains 91% of the item variance.
#> label difficulty se p.value
#> meaning -1.53 0.13 <0.001
#> equivalent -0.43 0.13 <0.001
#> mixed -0.39 0.11 <0.001
#> compare 0.05 0.13 0.68
#> common_denominator 0.01 0.12 0.96
#> add_subtract 0.67 0.12 <0.001
#> format: open (vs choice) -0.03 0.09 0.74
#> Items the graph predicts poorly: F22 (-0.59).
# }