Summarizes the calibrated difficulties of items generated from the same item model (a family of isomorphs): The family's mean difficulty, how much its items vary around it, and whether the family mean could stand in for field testing a new isomorph.
Usage
calibrate_families(items, family = "model", b = "b", se = NULL, max_sd = 0.3)
# S3 method for class 'mv_families'
autoplot(object, ...)Arguments
- items
a data frame with one row per calibrated item.
- family
name of the column with the item model of each item.
- b
name of the column with the difficulties.
- se
optional name of the column with the standard errors of
b.- max_sd
largest within-family SD, in logits, for a family to count as isomorphic.
- object
a result of
calibrate_families().- ...
not used.
Value
A tibble of class mv_families with one row per family:
family, n_items, mean_b, sd_b (observed), sd_within
(corrected for estimation error when se is given), se_mean,
lower and upper (a 95% interval for a new isomorph), field_test_n
and isomorphic. It has print() and autoplot() methods.
Details
The observed spread of the difficulties in a family includes their
estimation error. With se, the error variance is subtracted, which
leaves the within-family SD: How much isomorphs really differ. For a
new isomorph, the family mean predicts its difficulty with a typical error
of about the within-family SD (plus the uncertainty of the mean). The
result also translates that error into the number of test takers a field
test would need to estimate the item's difficulty as precisely
(field_test_n, for a Rasch item and abilities from a standard normal
distribution).
A family whose within-family SD exceeds max_sd is not isomorphic: Some
variable in the item model changes what the items demand. Look for it
among the values the items were generated from, and split the model or
add the variable as a feature (see predict_difficulty()).
This is the two-step approach. A one-step alternative fits the responses
with the family as a fixed effect and a random effect for items, for
example with mirt's mirt::mixedmirt() (fixed = ~ 0 + family, random = ~ 1 | items) or with the eirm package; see the examples in the online
book's chapter on automatic item generation.
Examples
cal <- calibrate_families(itemtext[itemtext$a > 0, ], family = "family", se = "b_se")
cal
#> Item families: 12 families, 189 items; SD within corrected for estimation error
#> family items mean_b sd_within 95% for new item field_n isomorphic
#> Converting units 16 -0.74 0.19 -1.13 to -0.36 148 yes
#> Area of a rectangle 16 -0.56 0.34 -1.23 to 0.12 45 NO
#> Mean 15 -0.45 0.18 -0.81 to -0.10 161 yes
#> Unit rate 16 -0.15 0.27 -0.69 to 0.40 66 yes
#> Average speed 16 0.16 0.27 -0.37 to 0.70 69 yes
#> Percent of a number 16 0.34 0.17 -0.01 to 0.69 165 yes
#> Elapsed time 16 0.39 0.18 0.03 to 0.75 156 yes
#> Probability 15 0.72 0.15 0.41 to 1.04 222 yes
#> Ratio 15 0.85 0.19 0.46 to 1.25 143 yes
#> Adding fractions 16 0.87 0.26 0.36 to 1.39 83 yes
#> Comparing decimals 16 1.11 0.19 0.73 to 1.50 162 yes
#> Solving equations 16 1.15 0.17 0.81 to 1.49 208 yes
#> field_n: Test takers a field test would need to estimate a new isomorph's b as precisely
#> as its family mean predicts it. Isomorphic: SD within at most 0.30.
autoplot(cal)