smartcor: Intelligent Correlation Method Selection for Mixed Variable Types

2026-07-24

AI Summary

This study addresses the common misuse of Pearson correlation for mixed variable types—such as binary, ordinal, and nominal—which often introduces bias in traditional correlation analyses. To resolve this, the authors introduce smartcor (for R) and pysmartcor (for Python), the first toolkits to systematically support all ten possible combinations of variable types. These packages employ automatic variable-type detection and a rule-based engine to intelligently select the optimal correlation or association method from a repertoire of fourteen, while also providing interpretable justifications for each choice. Monte Carlo simulations demonstrate substantially improved selection accuracy, and real-world case studies reveal meaningful discrepancies between type-aware analyses and naive Pearson correlations, thereby enhancing the reliability of statistical inference.

Abstract

Pearson correlation is the default measure of association in most statistical software, yet it is only appropriate for pairs of continuous variables with a linear relationship. When variables are binary, ordinal, or categorical, specialized methods (e.g., point-biserial, polychoric, tetrachoric, and Cramér's~$V$) may be more appropriate, but practitioners rarely know which to select. The \textbf{smartcor} package for \textbf{R} (and its companion \textbf{pysmartcor} package for \textbf{Python}) automatically detects variable types, selects the statistically appropriate correlation method for each pair, and explains its reasoning. The package supports 14~correlation and association methods covering all 10~variable-type pair combinations, and distinguishes true correlation (for ordinal and continuous pairs) from statistical association (for nominal categorical pairs). Monte~Carlo simulations validate the selection logic, and a case study with General Social Survey data demonstrates substantive differences between naive and type-aware correlation analysis.

Problem

• correlation
• variable types
• statistical association
• mixed data
• method selection

Innovation

• smartcor
• variable-type-aware correlation
• automatic method selection
• mixed variable types
• statistical association