A Simulation Study to Compare Inferential Properties when Modelling Ordinal Outcomes: The Case for the (Plain but Robust) Proportional Odds Model

2026-03-02

AI Summary

This study addresses regression modeling for ordinal outcomes in psychology and social behavioral sciences by systematically comparing the inferential performance of five model classes—proportional odds, partial proportional odds (category-specific), location-shift, location-scale, and misapplied linear models—under diverse data-generating mechanisms within a unified Monte Carlo simulation framework. Emphasizing parameter estimation bias, Type I error control, and statistical power, this work provides the first comprehensive quantification of the robustness and reliability of these approaches. The proportional odds model consistently demonstrates low bias, well-controlled Type I error rates, and high power across most scenarios, maintaining stable performance even under high skewness or large effect sizes, thereby establishing it as the recommended default method for analyzing ordinal outcomes.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Cognitive Modeling & Cognitive Systems: Analogy
• Machine Learning: Learning Preferences or Rankings

Application Category

• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Ordinal measurements are common outcomes in studies within psychology, as well as in the social and behavioral sciences. Choosing an appropriate regression model for analysing such data poses a difficult task. This paper aims to facilitate modeling decisions for quantitative researchers by presenting the results of an extensive simulation study on the inferential properties of common ordinal regression models: the proportional odds model, the category-specific odds model, the location-shift model, the location-scale model, and the linear model, which incorrectly treats ordinal outcomes as metric. The simulations were conducted under different data generating processes based on each of the ordinal models and varying parameter configurations within each model class. We examined the bias of parameter estimates as well as type I error rates ($α$-errors) and the power of statistical parameter testing procedures corresponding to the respective models. Our findings reveal several highlights. For parameter estimates, we observed that cumulative ordinal regression models exhibited large biases in cases of large parameter values and high skewness of the outcome distribution in the true data generation process. Regarding statistical hypothesis testing, the proportional odds model and the linear model showed the most reliable results. Due to its better fit and interpretability for ordinal outcomes, we recommend the use of the proportional odds model unless there are relevant contraindications.

Problem

• ordinal outcomes
• regression models
• model selection
• inferential properties
• proportional odds model

Innovation

• ordinal regression
• proportional odds model
• simulation study
• inferential properties
• type I error