Development and performance of npd for the evaluation of models with ordinal data

2026-05-04

AI Summary

This study addresses the limitation of the normalized prediction distribution error (NPDE), which is currently applicable only to continuous response variables and thus unsuitable for evaluating models with ordinal categorical outcomes. The authors extend NPDE to the ordinal setting by employing a jittering technique to generate approximately continuous residuals and assessing their standard normality via the Kolmogorov–Smirnov test. The proposed approach facilitates model diagnostics under unbalanced designs and incorporates visualization tools to evaluate covariate effects. Demonstrated through Monte Carlo simulations and a real-world clinical trial dataset on toenail onychomycosis, the method effectively identifies misspecifications in structural or parameter assumptions, with detection power increasing as distributional discrepancies and sample sizes grow. Graphical outputs clearly reveal biases in baseline models and confirm the improved fit of refined specifications.

Abstract

Introduction: Normalised prediction distribution errors (npde) are used to graphically and statistically evaluate continuous responses in non-linear mixed effect models. Here, our aim was to extend npde for categorical data and to evaluate their performance. We applied our approach to a real case-study describing the evolution of severe onychomycosis (toenail infection) in a trial comparing two treatment groups. Methods: Let V denote a dataset with categorical observations. The null hypothesis H0 is that observations in V can be described by a model M. Residuals called npde can be adapted to categorical observations using jittering techniques. Their theoretical standard normal distribution can be evaluated through the Kolmogorov-Smirnov test. We evaluated the performance in terms of power through a simulation and compared it to a Chi-square. We illustrated the test and graphs on a real case-study. Results: npd were able to detect misspecifications in the structural model and model parameter value. As expected, the power to detect model misspecifications increased both with the difference in the shape of the probability, and with the sample size. Chi-square test performed better but npd could be readily applied in all type of design. Based on the toe-nail data, graphs reveal a huge discrepancy of the base model, and a good adequation for the best model we found. Conclusions: npde can be extended to categorical data, particularly in clinical settings with unbalanced design and graphs can be useful to evaluate the model as well as the covariate effects.

Problem

• npde
• ordinal data
• model evaluation
• non-linear mixed effect models
• categorical data

Innovation

• npde
• ordinal data
• jittering
• model evaluation
• non-linear mixed effects models