🤖 AI Summary
Conventional categorical encoding methods (e.g., one-hot) in industrial process modeling lack semantic expressiveness, failing to capture meaningful relationships among categories such as reactor types or operation sequences. Method: This paper introduces, for the first time, an NLP-inspired semantic-aware categorical embedding framework: category-specific semantic vectors are generated via pre-trained language models and subsequently projected into an interpretable low-dimensional space using PCA or UMAP. Contribution/Results: Unlike conventional encodings, the proposed method explicitly models semantic distances between categories and enables quantitative feature importance analysis. Evaluated on an industrial case study involving cutting tool coatings, it achieves significant predictive performance gains. Moreover, it natively supports heterogeneous inputs—integrating both categorical and numerical features—thereby overcoming a fundamental limitation of existing encoding paradigms that cannot represent categorical similarity.
📝 Abstract
Important variables of processes are often categorical, i.e. names or labels representing, e.g. categories of inputs, or types of reactors or a sequence of steps. In this work, we use Natural Language Processing Models to derive embeddings of such inputs that represent their actual meaning, or reflect the"distances"between categories, i.e. how similar or dissimilar they are. This is a marked difference from the current standard practice of using binary, or one-hot encoding to replace categorical variables with sequences of ones and zeros. Combined with dimensionality reduction techniques, either linear such as Principal Component Analysis, or nonlinear such as Uniform Manifold Approximation and Projection, the proposed approach leads to a meaningful, low-dimensional feature space. The significance of obtaining meaningful embeddings is illustrated in the context of an industrial coating process for cutting tools that includes both numerical and categorical inputs. In this industrial process, subject matter expertise suggests that the categorical inputs are critical for determining the final outcome but this cannot be taken into account with the current state-of-the-art. The proposed approach enables feature importance which is a marked improvement compared to the current state-of-the-art in the encoding of categorical variables. The proposed approach is not limited to the case-study presented here and is suitable for applications with similar mix of categorical and numerical critical inputs.