Deconstructing the Crystal Ball: From Ad-Hoc Prediction to Principled Startup Evaluation with the SAISE Framework

2025-08-07

AI Summary

Existing AI-driven startup evaluation research suffers from methodological fragmentation: inconsistent definitions of success, theoretically unsupported feature selection, and insufficient validation rigor—undermining model comparability, reliability, and practical utility. To address this, we systematically review 57 studies and identify four core deficiencies: heterogeneous success conceptualization, ad hoc feature construction, inadequate validation protocols, and neglect of ethical interpretability. Building on this analysis, we propose SAISE—a novel five-stage, theory- and data-integrated framework encompassing Problem Formulation, Data Fusion, Feature Engineering, Validation, and Risk Interpretation. Through systematic literature analysis, cross-study feature mapping, and algorithmic pattern synthesis, we uncover convergent trends toward venture capital databases and tree-based ensemble models, while exposing persistent methodological divergences. The SAISE framework establishes a more comparable, robust, and practice-oriented research paradigm for AI-enabled startup assessment.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

The integration of Artificial Intelligence (AI) into startup evaluation represents a significant technological shift, yet the academic research underpinning this transition remains methodologically fragmented. Existing studies often employ ad-hoc approaches, leading to a body of work with inconsistent definitions of success, atheoretical features, and a lack of rigorous validation. This fragmentation severely limits the comparability, reliability, and practical utility of current predictive models. To address this critical gap, this paper presents a comprehensive systematic literature review of 57 empirical studies. We deconstruct the current state-of-the-art by systematically mapping the features, algorithms, data sources, and evaluation practices that define the AI-driven startup prediction landscape. Our synthesis reveals a field defined by a central paradox: a strong convergence on a common toolkit -- venture databases and tree-based ensembles -- but a stark divergence in methodological rigor. We identify four foundational weaknesses: a fragmented definition of "success," a divide between theory-informed and data-driven feature engineering, a chasm between common and best-practice model validation, and a nascent approach to data ethics and explainability. In response to these findings, our primary contribution is the proposal of the Systematic AI-driven Startup Evaluation (SAISE) Framework. This novel, five-stage prescriptive roadmap is designed to guide researchers from ad-hoc prediction toward principled evaluation. By mandating a coherent, end-to-end methodology that emphasizes stage-aware problem definition, theory-informed data synthesis, principled feature engineering, rigorous validation, and risk-aware interpretation, the SAISE framework provides a new standard for conducting more comparable, robust, and practically relevant research in this rapidly maturing domain

Problem

• Fragmented AI methods in startup evaluation research
• Inconsistent success definitions and weak validation
• Lack of standardized framework for predictive models

Innovation

• Systematic AI-driven Startup Evaluation (SAISE) Framework
• Five-stage prescriptive roadmap for evaluation
• Emphasizes theory-informed data synthesis