new_lab() writes an interactive lab (a learnr tutorial) that walks
students through an IRT analysis of your dataset: The data, classical
item statistics, dimensionality, model comparison, item fit, information,
scoring and, when you give a grouping variable, DIF. With item_info or
scores it can also teach AI in measurement on your data: What the item
text says about difficulty, how equally difficult the items of each item
family are, and whether machine scores agree with human scores. Every number in the
text, every answer check and every multiple-choice question is computed
from the data when the lab is rendered, so the same lab works with any
dataset and swap_lab_data() can replace the data later without touching
the text.
Usage
new_lab(
name,
data,
course = ".",
items = NULL,
group = NULL,
key = NULL,
item_info = NULL,
scores = NULL,
title = NULL,
data_title = NULL,
story = NULL,
sections = NULL,
overwrite = FALSE
)Arguments
- name
the lab's name: Letters, digits,
-and_.- data
the responses: A data frame or matrix, or the path of a file (
.csv,.tsv,.txt,.xlsx,.xls,.sav,.dta,.sas7bdator.rds). Items must be scored 0/1 or in ordered integer categories, unless you give akey. A frequency table (one row per response pattern with afreqorcountcolumn, such as mirt::LSAT7) is expanded to one row per person.- course
the course folder (see
new_course()); the lab goes in itslabs/folder.- items
the item columns, by name or position. By default, every column whose values are whole numbers from 0 to 9 (other than
groupand ID-like columns), or the names of a namedkey.- group
optional grouping variable with two values, for the DIF section: A column name of
data, or a vector with one value per row.- key
optional answer key for unscored multiple-choice responses, one value per item; the items are scored with
mirt::key2binary().- item_info
optional data frame (or the path of a data file) with one row per item, for the AI sections. It needs a column
item(orid) with the item names used indata. Astemcolumn (and, for multiple-choice items, option columnsAtoEand akeycolumn with the letter of the keyed option) enables the"item-features"section; afamilycolumn naming each item's item model or family enables the"aig"section.- scores
optional data frame (or the path of a data file) of machine scores of constructed responses, for the
"ai-scoring"section: One row per scored response, with columnsmachineandhuman1(the human score,NAwhere no person read the response) and, optionally,human2(a second reading),group(for fairness checks),itemandperiod(an administration or engine version, for drift checks). Rename your columns to these names first.- title
the lab's title.
- data_title
a short name for the data used in the lab's text, for example
"the Grade 8 science test".- story
a paragraph that sets the scene, shown at the start of the lab.
- sections
the sections to include, in order; see
lab_section_types(). The default includes every section that suits the data:"dif"only with agroup, and the AI sections only whenitem_infoorscoresprovide what they need.- overwrite
replace an existing lab of the same name.
Details
The lab is a folder labs/<name>/ inside the course folder, holding
<name>.Rmd and a data/ folder (responses.rds and lab.json). Edit
the .Rmd freely; add_lab_section(), add_lab_exercise() and
add_lab_question() add content, and check_material() runs every
exercise's solution through its check.
Examples
course <- file.path(tempdir(), "lab-demo")
new_course(course)
lab <- new_lab("lsat", data = expand.table(LSAT7), course = course,
data_title = "the LSAT section 7 items",
sections = c("intro", "classical", "models", "wrap-up"))
lab_outline(lab)
#> # A tibble: 4 × 5
#> id type title exercises questions
#> <chr> <chr> <chr> <int> <int>
#> 1 intro intro Meet the data 1 0
#> 2 classical classical Classical item statistics 2 1
#> 3 models models Compare IRT models 2 1
#> 4 wrap-up wrap-up Where to next 0 0
# an AI lab: item text and item families of 60 generated items
lab2 <- new_lab("aig", data = aigfield, course = course,
item_info = transform(aigfield_items, family = model),
data_title = "the generated items",
sections = c("intro", "item-features", "aig", "wrap-up"))
lab_outline(lab2)
#> # A tibble: 4 × 5
#> id type title exercises questions
#> <chr> <chr> <chr> <int> <int>
#> 1 intro intro Meet the data 1 0
#> 2 item-features item-features What the item text tells us 2 3
#> 3 aig aig Item families 2 2
#> 4 wrap-up wrap-up Where to next 0 0