Choosing a model is the first real decision in an IRT analysis, and
the one a committee or a technical advisory panel asks about first. The
flowchart below starts from what a single item score looks like, asks
one or two questions, and ends at a model and the itemtype
that fits it in mirt.
Download the poster (PDF) Open it as an image
The poster is US Letter, landscape, and prints well in colour or greyscale. The QR code links back to this page. Feel free to put it on a classroom wall, in a course pack or on a conference poster.
Not sure? Let the data compare
If the questions do not settle it, fit the usual candidates and
compare them. select_model() picks the candidates from the
data (Rasch, 2PL and 3PL for 0/1 items; partial credit, generalized
partial credit and graded for ordered items), ranks them by BIC, runs
likelihood-ratio tests between nested models, and explains its pick:
library(mirtverse)
sel <- select_model(expand.table(LSAT7))
sel # comparison table and a plain-language explanation
mod <- sel$best
interpret(mod)The statistics are only half of the decision. The purpose of the test matters too: A Rasch model can be the right choice even when a 2PL fits a little better, because it keeps the sum score as the whole story.
Right or wrong (0/1) items
Could low scorers guess the right answer? On multiple-choice items with a few options, they often can. The 3PL adds a lower asymptote for guessing, but it needs large samples (about 1,000 people or more) to estimate well.
data <- expand.table(LSAT7)
mod3 <- mirt(data, 1, itemtype = "3PL")Should every item count the same in the score? If yes, use the Rasch model: All items share one slope, the sum score carries all the information about ability, and it works with smaller samples (from about 100 people). If items should be allowed to discriminate differently, use the 2PL (aim for about 500 people or more).
Ordered categories
Are points earned step by step on a rubric? For
constructed responses and tasks scored 0, 1, 2 and so on, the
generalized partial credit model treats each score as a step. Its Rasch
version, the partial credit model, is itemtype = "Rasch" in
mirt when items have more than two categories.
Same rating scale on every item, and a Rasch model wanted? The rating scale model gives all items the same set of thresholds, which makes it the most compact choice. Otherwise, the graded response model is the usual pick for Likert scales.
Unordered options
When you care about which wrong option a person chose, keep the raw
responses and fit the nominal response model. It shows which distractors
attract lower-ability test takers and which ones do not work.
SAT12 stores the raw option chosen (1 to 5) for 32
items.
Agree only if close (unfolding)
Some attitude statements describe a middle position, such as “Social media is fine in moderation”. People on both extremes disagree with it, for opposite reasons. Ideal-point (unfolding) models handle this, and the generalized graded unfolding model (GGUM) is available in mirt:
mod_ggum <- mirt(Science, 1, itemtype = "ggum")Then think about the structure
Each of these questions adds one argument or one function to the model you picked above.
| Question | What to do | Learn more |
|---|---|---|
| More than one trait? | Check local dependence with gg_q3(mod), then fit
mirt(data, 2) or a confirmatory
mirt.model()
|
Glossary: Dimensionality |
| Items in clusters (testlets, shared passages)? | A general factor plus specific factors with
bfactor(data, specific)
|
?bfactor |
| Groups to compare? | Screen items with dif(mod, group), then put groups on
one scale with multipleGroup()
|
Lab 3: DIF |
| Item features explain difficulty? | Explanatory IRT (LLTM) with predict_difficulty(), the
eirm package or
mixedmirt()
|
Book: Item text |
| Response times too? | Model speed together with ability with
fit_rt(mod, rt)
|
Response times |
| Classes rather than a continuum? | Mastery groups or latent classes with
mdirt(data, 2)
|
?mdirt |
Before you trust the model
Whatever path you took, check the result before you report it:
glance(mod) # overall fit: M2, RMSEA, CFI, SRMSR
gg_itemfit(mod) # items that misfit
gg_trace(mod) # curves that look strange
interpret(mod) # a plain-language summaryA simpler model that fits is easier to explain. Move to a richer one only when the fit checks or the purpose of the test call for it. The thesis kit turns these checks into a Methods paragraph.
A note on sample sizes
The sample sizes on the poster are rough rules of thumb drawn from
the IRT literature, not guarantees. More items, well-targeted items and
simpler models all need fewer people; the 3PL and the nominal model need
the most. When in doubt, simulate: simdata() in mirt can
show how well your planned sample recovers the parameters.
