SPOCK 2.0: Update to the FeatureClassifier in the Stability of Planetary Orbital Configurations Klassifier

2025-01-25

AI Summary

The SPOCK planetary orbital stability classifier suffers from limited prediction accuracy due to training data contamination—approximately 10% duplicated samples and <1% mislabeled stable instances. Method: We replace the original fixed-duration simulation (10⁴ orbits) with a systematic adaptive integration time mechanism, incorporating this dynamically determined duration as a key feature. Leveraging N-body numerical integration, rigorous data curation, and XGBoost-based machine learning, we construct a new high-quality dataset exceeding 100,000 samples, then retrain and publicly release an updated model and API. Contribution/Results: Our work introduces the first orbit-duration adaptive feature engineering framework, significantly enhancing stability classification performance under short-duration simulations: AUC improves from 0.943 to 0.950. The revised model exhibits improved robustness and stronger astrophysical interpretability, aligning more closely with underlying dynamical principles.

Technology Category

• Machine Learning: Feature Construction/Reformulation
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Web data integration and cleaning
• Security and Privacy: Data transparency and provenance

Abstract

The Stability of Planetary Orbital Configurations Klassifier (SPOCK) package collects machine learning models for predicting the stability and collisional evolution of compact planetary systems. In this paper we explore improvements to SPOCK's binary stability classifier (FeatureClassifier), which predicts orbital stability by collecting data over a short N-body integration of a system. We find that by using a system-specific timescale (rather than a fixed $10^4$ orbits) for the integration, and by using this timescale as an additional feature, we modestly improve the model's AUC metric from 0.943 to 0.950 (AUC=1 for a perfect model). We additionally discovered that $approx 10%$ of N-body integrations in SPOCK's original training dataset were duplicated by accident, and that $<1%$ were misclassified as stable when they in fact led to ejections. We provide a cleaned dataset of 100,000+ unique integrations, release a newly trained stability classification model, and make minor updates to the API.

Problem

• Machine Learning
• Orbital Stability
• Data Quality

Innovation

• SPOCK 2.0
• Customized Observation Time
• Enhanced Dataset