Skip to contents

59,611 synthetic respondents to the 240 IPIP-HEXACO items, generated by four open-weight large language models for Canada, Great Britain and the United States, in 10 independent attempts of about 500 respondents per model and country. They come from a study of how well synthetic respondents cover real people, and they pair with the real answers in hexaco_real.

Usage

hexaco_llm

Format

A data frame with 59,611 rows (one per synthetic respondent) and 243 columns: model (factor, the four models above), country (factor: Canada, Great Britain or United States), attempt (1 to 10) and the 240 items, named as in hexaco_items$item.

Source

Generated for the coverage strip study by Okan Bulut. The script is data-raw/hexaco.R in the package's GitHub repository.

Details

Each respondent was generated in its own conversation. The model was asked to simulate one adult drawn at random from the general adult population of the named country (not an average or typical person), to answer as that person rather than as an AI or a socially desirable person, and to use the full range of the scale. No demographic details were given, so the country was the only part of the prompt that changed. The items were shown in their original order in six blocks of 40, at temperature 1.0, between July and September 2026. The models ran on the Aleph Inference Gateway of the Vulcan cluster (Digital Research Alliance of Canada):

modelCheckpointDeveloper
Gemma 4 26Bgoogle/gemma-4-26B-A4B-it (FP8)Google
GPT-OSS-120Bopenai/gpt-oss-120b (MXFP4)OpenAI
Qwen 3.8 27BQwen/Qwen3.8-27B-FP8Alibaba
Qwen 3.5 122BQwen/Qwen3.5-122B-A10B-FP8Alibaba

379 rows that were logged under another model in the Qwen 3.8 27B files were removed, so a few Qwen 3.8 27B attempts have fewer than 500 respondents. As in hexaco_real, answers are stored as given (1 to 7), with reverse-keyed items not reversed. No answer is missing.

Examples

with(hexaco_llm, table(model, country))
#>                country
#> model           Canada Great Britain United States
#>   Gemma 4 26B     5000          5000          5000
#>   GPT-OSS-120B    5000          5000          5000
#>   Qwen 3.8 27B    4945          4995          4671
#>   Qwen 3.5 122B   5000          5000          5000
# one list per model, one data frame per attempt: the shape that
# synthetic_coverage() takes
h <- hexaco_items$item[hexaco_items$trait == "H"]
ca <- hexaco_llm[hexaco_llm$country == "Canada", ]
syn <- lapply(split(ca, ca$model), function(m) split(m[h], m$attempt))
lengths(syn)
#>   Gemma 4 26B  GPT-OSS-120B  Qwen 3.8 27B Qwen 3.5 122B 
#>            10            10            10            10