This page is a suggested 12-week syllabus for an introductory course in item response theory (IRT), for graduate students in education, psychology or the health sciences. Each week lists what students should be able to do by the end of it, what to read on this site, what to run in R, and which lab and dataset to use. Take it as a starting point: drop weeks, merge them, or swap in your own readings.
Everything here runs on a laptop with mirt and mirtverse installed. No internet connection is needed once the packages are installed, and none of the examples take more than a few seconds to run.
Before the first class
Ask students to install the packages and check that a lab opens:
install.packages(c("mirt", "learnr"))
# install mirtverse as shown on the home page, then:
library(mirtverse)
list_labs()Students who are new to R can do much of the course without writing
code. The workbench
(launch_workbench()) runs a full analysis by pointing and
clicking and shows the R code for every step, and mv_ask() writes a script from
a plain-language request. Both make good bridges to writing code by hand
later in the term.
The 12 weeks at a glance
| Week | Topic | Lab | Main data |
|---|---|---|---|
| 1 | Measurement, scoring and classical item analysis | Lab 1, first two sections | certexam |
| 2 | From classical test theory to IRT: the Rasch, 2PL and 3PL models | LSAT7 |
|
| 3 | Estimation and comparing models |
LSAT7, SAT12
|
|
| 4 | Dimensionality and local independence | Lab 1 in class | certexam |
| 5 | Item fit, person fit and item quality | SAT12 |
|
| 6 | Information, precision and reliability | certexam |
|
| 7 | Scoring people | Lab 2 in class | certexam |
| 8 | Polytomous items and open-ended responses | Lab 6, run_lab("open-ended")
|
openended, Science
|
| 9 | Differential item functioning and fairness | Lab 3, run_lab("fairness")
|
numeracy, certexam
|
| 10 | Linking and equating | Lab 4, run_lab("linking")
|
linking, certexam
|
| 11 | Item banks, test assembly and adaptive testing | Lab 5, run_lab("cat-design")
|
catbank, certexam bank |
| 12 | Cut scores, reporting and communicating results | Final project | students’ own data |
The glossary defines each term used below, and each week links to its key terms there. The cheat sheet lists the main functions on two printable pages; both work well as handouts from week 1.
Week by week
Week 1. Measurement, scoring and classical item analysis
By the end of the week, students can score multiple-choice responses against a key, compute proportion correct and item-total correlations, and read a distractor analysis.
- Key terms: classical test theory, sum score, p-value, item-total correlation.
- Read: Getting started; “The story” and “Start a lab” in Teaching labs.
-
Run:
key2binary(),itemstats()(mirt),profile_data(). -
Lab: the first two sections of Lab 1
(
run_lab("model-choice")), “Score the responses” and “Screen the items”. Students meet a miskeyed item without being told it is there. -
Data:
certexam,certexam_key. - Discuss: what a number-correct score can and cannot tell us.
Week 2. From classical test theory to IRT
By the end of the week, students can explain the
Rasch, 2PL and 3PL models, fit them with mirt(), and read
an item characteristic curve.
- Key terms: item response theory, latent trait, Rasch model, 2PL, 3PL, item characteristic curve, difficulty, discrimination, guessing.
- Read: sections 1 to 4 of Getting started.
-
Run:
mirt(),coef(mod, IRTpars = TRUE),gg_trace(),item_table(),interpret(). -
Data:
LSAT7(expand it withexpand.table()). - Activity: ask students to sketch trace lines for items they invent (an easy item, a hard item, a flat item) and then find the closest real item in the data.
Week 3. Estimation and comparing models
By the end of the week, students can describe marginal maximum likelihood in words, recognise a model that has not converged, and compare nested models with likelihood-ratio tests and information criteria.
- Key terms: calibration, marginal maximum likelihood, convergence, likelihood-ratio test, information criteria, M2, RMSEA.
- Read: “Let mirtverse pick a model” in Getting started.
-
Run:
select_model(),anova(),M2(),glance_mirt(),tidy_mirt(). -
Data:
LSAT7andSAT12(scored withkey2binary()). - Discuss: why AIC and BIC can disagree, and what sample size has to do with fitting a 3PL.
Week 4. Dimensionality and local independence
By the end of the week, students can check whether one dimension is enough, read standardized residuals and the Q3 statistic, and tell local dependence from a second trait.
- Key terms: dimensionality, local independence, local dependence, Q3.
-
Run: a one- vs two-factor model with
anova(),summary()for rotated loadings,residuals(mod, type = "Q3")andgg_q3(). -
Lab: all of Lab 1
(
run_lab("model-choice"), about 75 minutes). It ends with a model choice the student has to justify in writing. -
Data:
certexam(the passage pair Q21 and Q22). - Assignment: hand in the decision table and justification from the end of Lab 1.
Week 5. Item fit, person fit and item quality
By the end of the week, students can read S-X² item fit, compare an empirical and a model-implied curve, and flag aberrant response patterns.
- Key terms: item fit, S-X², empirical item curve, person fit, Zh.
-
Run:
itemfit()andgg_itemfit(),gg_empirical(),personfit()(mirt),item_card()anditem_cards(). -
Data:
SAT12. - Activity: students pick the three weakest items from the item cards and write one sentence each on what an item writer should change.
Week 6. Information, precision and reliability
By the end of the week, students can read item and test information curves, convert information to a standard error, and explain why reliability depends on where on the scale you look.
- Key terms: item information, test information, standard error of measurement, marginal reliability, targeting.
-
Run:
gg_info(),testinfo(),marginal_rxx()andempirical_rxx()(mirt). - Lab: the “Precision at the pass mark” section of Lab 2 previews next week.
-
Data:
certexam.
Week 7. Scoring people
By the end of the week, students can compare EAP, MAP, ML and WLE scores, explain why ML fails for perfect scores, and read a Wright map.
- Key terms: EAP, MAP, ML, WLE, Wright map, classification accuracy.
-
Run:
fscores()(mirt),gg_scores(),gg_wright(),augment_mirt(). -
Lab: all of Lab 2
(
run_lab("decisions"), about 75 minutes). Students choose a scoring method, estimate classification accuracy, look for DIF and write a memo to the board. Lab 2 fits its own model, so students who missed Lab 1 can still do it. -
Data:
certexam. - Assignment: the memo from the end of Lab 2.
Week 8. Polytomous items and open-ended responses
By the end of the week, students can fit the partial credit, generalized partial credit and graded response models, read category curves, and say when categories should be collapsed.
- Key terms: polytomous item, graded response model, GPCM, category response curve, QWK.
-
Read: Assessment
pipeline, section “9. Scoring”, for scoring open-ended responses
with
score_open(). -
Run:
select_model()on polytomous data,gg_trace(),gg_info(). -
Lab: Lab 6 (
run_lab("open-ended"), about 75 minutes): rater agreement and quadratic weighted kappa, rater severity and consistency, resolving two reads, PCM versus GPCM versus GRM, collapsing categories with disordered steps, and checking an automated scoring engine, including for subgroups. -
Data:
openended(six rater-scored science tasks) in Lab 6;Science(four Likert-type items) for extra practice. -
Students coming from ltm, eRm or TAM:
mirtify()translates their old code (Coming from other IRT packages).
Week 9. Differential item functioning and fairness
By the end of the week, students can tell impact from DIF, run a DIF analysis with anchor items, and explain uniform and non-uniform DIF.
- Key terms: DIF, reference and focal groups, uniform and non-uniform DIF, DIF effect size, anchor items.
-
Run:
dif(),gg_dif(),interpret()on the result;multipleGroup()andDIF()(mirt) for the model behind it. -
Lab: the “Fairness across languages” section of Lab
2 for a short version; Lab 3 (
run_lab("fairness"), about 75 minutes) for a full session, with Mantel-Haenszel, anchor purification and differential test functioning. -
Data:
numeracy(English language learners and fluent students) in Lab 3;certexam(English and French candidates) in Lab 2. - Discuss: what a content review panel should do with a flagged item.
Week 10. Linking and equating
By the end of the week, students can explain why two forms need a common scale, describe anchor-item and common-person designs, and link a new calibration to an existing one.
- Key terms: linking, anchor items, moment linking, characteristic curve linking, parameter drift.
- Read: Assessment pipeline, sections “3. Field-test design” and “6. The item bank”.
-
Run:
design_field_test(),link_to_bank();multipleGroup()with anchors andfixedCalib()(mirt). -
Lab: Lab 4 (
run_lab("linking"), about 75 minutes): separate and concurrent calibration, mean-sigma linking, anchor drift and IRT true-score equating of a pass mark. -
Data:
linking(two forms with common items) in Lab 4;certexam, split into two field-test forms bydesign_field_test(), for the pipeline functions.
Week 11. Item banks, test assembly and adaptive testing
By the end of the week, students can describe an item bank, assemble a form to a target information curve, and explain how a computerized adaptive test picks items and when it stops.
- Key terms: item bank, automated test assembly, computerized adaptive test, multistage test, exposure.
- Read: Assessment pipeline, sections “6. The item bank” and “7. Forms, adaptive and multistage tests”.
-
Run:
mv_bank(),assemble_forms(),simulate_cat(),assemble_mst(). -
Lab: Lab 5 (
run_lab("cat-design"), about 75 minutes): bank information, a CAT by hand, simulation, exposure control, content balancing and stopping rules. -
Data:
catbank(a 180-item placement bank) in Lab 5; thecertexambank built in the pipeline article for the pipeline functions.
Week 12. Cut scores, reporting and communicating results
By the end of the week, students can describe a Bookmark standard setting, estimate classification accuracy at a cut score, and explain IRT results to someone who does not use R.
- Key terms: cut score, standard setting, response probability, classification accuracy, classification consistency.
- Read: Assessment pipeline, sections “8. Cut scores” and “11. The report”; Sharing results with a dashboard.
-
Run:
item_map(),set_cuts(),mv_report(),mv_dashboard(). - Final project: see below.
Assessment ideas
| Weight | Task | When |
|---|---|---|
| 20% | Lab 1 decision table and model justification | Week 4 |
| 20% | Lab 2 memo to the board | Week 7 |
| 20% | Two short problem sets (one on fit and precision, one on DIF or linking) | Weeks 6 and 10 |
| 40% | Final project: a full analysis of a dataset of the student’s choice,
handed in as an mv_report() or a dashboard plus a two-page
memo |
Week 12 |
For the final project, students can follow the assessment pipeline from calibration to the report. Its flags point out what to check before each next step, which works well as a checklist for the write-up.
Adapting the course
- A shorter course (8 weeks): combine weeks 2 and 3, and weeks 5 and 6; drop week 10 and treat week 11 as a lecture without a lab.
- An undergraduate course: spend two weeks on week 1, use the workbench instead of code until week 4, and end at week 9.
-
A second course: add multidimensional and bifactor
models (
mirt()with more factors,bfactor()), response-time models (fit_rt(),sim_rt()), and the remaining stages of the assessment pipeline, such as test security and bank monitoring.
Planned resources
These resources are being built and will be linked from the weeks above when they are released.
| Resource | Used in | Status |
|---|---|---|
| Glossary of IRT terms on this site | All weeks | Available: glossary |
| Two-page cheat sheet of the main functions | All weeks | Available: PDF |
| DIF and fairness lab | Week 9 | Available: run_lab("fairness")
|
| Linking two forms lab | Week 10 | Available: run_lab("linking")
|
| CAT design lab | Week 11 | Available: run_lab("cat-design")
|
| Open-ended scoring lab | Week 8 | Available: run_lab("open-ended")
|