Multimodal Deep Learning for Joint Chip Morphology Classification and Tool Wear Prediction in Computer Numerical Control Milling

With the rapid development of intelligent manufacturing and Industry 4.0, increasing attention has been directed towards automated machining-process monitoring, intelligent tool-condition assessment, and chip-morphology analysis. In computer numerical control (CNC) milling, early monitoring approaches were primarily based on empirical machining theories, analytical force models, and statistical signal-processing techniques, with vibration and cutting-force measurements being widely employed for the detection of tool degradation and machining instability [1]. Although such approaches can provide reliable information under controlled laboratory conditions, their performance may deteriorate when cutting parameters, workpiece materials, or environmental conditions are varied. Moreover, conventional feature-engineering approaches are highly dependent on domain expertise and may be inadequate for representing the nonlinear interactions among machining parameters and heterogeneous sensor signals [2].

To overcome these limitations, machine-learning techniques, including support vector machines (SVMs), random forests (RFs), and eXtreme Gradient Boosting (XGBoost), have been increasingly employed for intelligent tool condition monitoring. Improved predictive performance has been reported because latent relationships between machining states and sensor-derived features can be learned directly from experimental data [3], [4]. Nevertheless, conventional machine-learning approaches generally remain dependent on manually designed features, which can limit their effectiveness when high-dimensional, heterogeneous, and temporally varying machining signals are considered. These limitations have encouraged the adoption of deep learning, through which more representative features can be learned directly from raw or minimally processed sensor data.

Convolutional neural networks (CNN) have consequently been widely applied to machining-signal analysis because local patterns and correlations can be extracted automatically from multidimensional signal representations [5]. Successful applications have been reported in tool-wear monitoring, chatter detection, and machining-fault diagnosis. However, CNN is primarily suited to local feature extraction and are not inherently designed to capture long-range temporal dependencies in sequential machining data. Recurrent neural networks, particularly long short-term memory (LSTM) and bidirectional LSTM (BiLSTM) networks, have therefore been employed to model temporal relationships in vibration, acoustic-emission (AE), and other sensor signals [6]. Improved performance has been reported for progressive tool-wear monitoring when CNN-based spatial feature extraction is combined with LSTM-based temporal modelling [7]. However, recurrent architectures can become computationally demanding for long sequences and are less amenable to parallel processing.

Transformer architectures have subsequently attracted increasing interest because their self-attention mechanisms allow relationships between distant elements in a sequence to be modelled without relying exclusively on recurrent processing. Their ability to selectively weight informative features and capture long-range dependencies has enabled their application to industrial time-series analysis, predictive maintenance, anomaly detection, and intelligent fault diagnosis [8], [9]. Despite these developments, the application of Transformer-based architectures to the simultaneous analysis of chip morphology and tool wear in CNC milling remains insufficiently explored. In particular, it remains unclear whether the complementary information contained in multiple machining-sensor modalities can be effectively integrated with spatial and temporal feature representations to improve the joint analysis of these two strongly coupled machining phenomena.

Multimodal sensor fusion provides a potential means of addressing this limitation. Modern CNC milling operations generate heterogeneous information through vibration, AE, cutting force, spindle current, and thermal measurements. The complementary characteristics of these signals can provide a more comprehensive representation of machining dynamics and can improve robustness against sensor noise and operating disturbances [10]. Multimodal learning has consequently been reported to provide more reliable predictions than approaches based on individual sensing modalities [11]. However, relatively simple fusion strategies have often been adopted, and the high-level interactions among heterogeneous sensor channels have not been fully exploited. More effective representations of cross-modal dependencies are therefore required, particularly when machining conditions vary substantially.

Chip morphology represents another important indicator of machining behaviour because chip formation is closely associated with cutting stability, heat generation, material deformation, and tool–workpiece interactions. Depending on the cutting conditions and material response, chips may exhibit continuous, segmented, serrated, or discontinuous morphologies, each of which may be associated with distinct characteristics in the acquired sensor signals [12]. Existing machine-learning approaches to chip classification have primarily relied on image-based analysis or individual sensor modalities. In many cases, chip morphology has been treated as an independent classification problem, while the simultaneous evolution of tool wear has received considerably less attention. Consequently, the potential relationship between chip morphology and progressive tool degradation has not yet been fully incorporated into unified multimodal monitoring frameworks.

Another important challenge concerns generalisation under variable machining conditions. Industrial milling processes are routinely performed under different spindle speeds, feed rates, lubrication conditions, and environmental disturbances. A model trained under a restricted set of operating conditions may therefore experience substantial performance degradation when applied to previously unseen conditions [13]. Robustness to such variations is particularly important for practical deployment, yet systematic evaluation under out-of-distribution machining conditions remains limited. In addition, interpretability, computational efficiency, and the effective exploitation of heterogeneous sensor interactions continue to represent important challenges for deep-learning-based machining monitoring.

Several research gaps can therefore be identified. First, most existing studies have focused primarily on either tool-wear monitoring or chip-morphology classification, while their coupled evolution during milling has rarely been addressed within a unified learning framework. Second, substantial reliance on handcrafted features remains evident in conventional machine-learning approaches, limiting their ability to learn adaptive representations from complex machining signals [14]. Third, cross-modal interactions among heterogeneous sensing channels have not been fully exploited in many existing deep-learning frameworks. Fourth, although Transformer-based architectures have demonstrated considerable potential for industrial time-series modelling, their integration with multimodal sensing and tool/chip analysis in CNC milling remains relatively limited. Finally, the robustness of existing models under variations in spindle speed, feed rate, and lubrication condition remains insufficiently established [15].

To address these gaps, a multimodal deep-learning (MM-DL) framework is developed for the simultaneous classification of chip morphology and prediction of tool wear during CNC milling. Heterogeneous signals from vibration, AE, cutting force, spindle current, and temperature measurements are integrated to construct a comprehensive representation of the machining state. Within the proposed framework, CNN-based feature extraction is employed to capture local signal characteristics, BiLSTM-based sequence modelling is used to characterise bidirectional temporal dependencies, and a Transformer-based self-attention mechanism is incorporated to capture long-range interactions among learned features. The resulting representation is used for both chip-morphology classification and progressive tool-wear prediction. Performance is further examined under different spindle speeds, feed rates, and wet and dry milling conditions, while comparative and ablation analyses are used to assess the contributions of the individual architectural components and sensing modalities.

To address the aforementioned challenges, a MM-DL framework is developed for the simultaneous classification of chip morphology and prediction of tool wear in CNC milling. The framework integrates multimodal sensor fusion with CNN-based local feature extraction, BiLSTM-based temporal modelling, and a Transformer-based self-attention mechanism. Heterogeneous signals acquired from vibration, AE, cutting force, spindle current, and temperature measurements are fused to provide a comprehensive representation of the machining process. Within the proposed architecture, local signal characteristics are extracted by the CNN, temporal dependencies are modelled by the BiLSTM, and long-range interactions among the learned features are captured by the Transformer encoder. The resulting multimodal representation is used to classify chip morphology and estimate progressive tool wear under varying machining conditions.

The main contributions are summarised as follows:

$\bullet$ A MM-DL framework is developed for the joint analysis of chip morphology and tool wear in CNC milling, enabling complementary information from heterogeneous sensing modalities to be exploited within a unified architecture;

$\bullet$ A hybrid CNN–BiLSTM–Transformer architecture is developed to extract local signal characteristics, model temporal dependencies, and capture long-range interactions in machining sensor data;

$\bullet$ The robustness of multimodal monitoring is evaluated under varying spindle speeds, feed rates, and wet and dry milling conditions, providing an assessment of model stability under different machining environments;

$\bullet$ The effectiveness of the proposed framework is systematically assessed through comparative and ablation experiments against conventional machine-learning and deep-learning approaches, with separate evaluation of chip-morphology classification and tool-wear prediction.

To solve the above problems, this paper proposes an intelligent multimodal learning architecture for chip formation and tool wear analysis in CNC milling. The proposed algorithm combines the advantages of multimodal sensor fusion, CNN-based spatial feature extraction, BiLSTM-based temporal sequence learning, and a transformer-based self-attention mechanism to classify the morphology of the chip and predict the wear of the tool simultaneously. To obtain a complete picture of the machining process, multiple heterogeneous sensor signals, such as vibration, AE, cutting force, spindle current, and temperature, are fused. The transformer encoder automatically discovers the critical feature of the cutting process and enhances the ability of the prediction to withstand changes in cutting conditions.

This work is summarized as follows:

$\bullet$ A new MM-DL structure is suggested to classify the chips and estimate the tool wear in the CNC milling process;

$\bullet$ To extract both local spatial features and long-range temporal dependencies in machining sensor signals, a hybrid CNN-BiLSTM-Transformer architecture is developed;

$\bullet$ Improved Prediction Robustness and Classification Stability under Varying Machining Conditions by Multimodal Sensor Fusion;

$\bullet$ The suggested spindle speed, feed rate, wet milling, and dry milling parameters are evaluated experimentally to determine the applicability of the suggested conditions in industry;

$\bullet$ Comparative and ablation analyses reveal that the proposed framework is superior to conventional machine learning and deep learning methods in classification and regression tasks.

The methodology that was used for the proposed Intelligent Multi-Modal Learning Architecture for Chip Formation and Tool Wear Analysis in CNC Milling is given. It combines multimodal sensor fusion, deep feature extraction, deep temporal learning with a transformer, and predictive analytics to simultaneously classify chip morphology and estimate tool wear.

The methodology developed for the proposed multimodal learning framework for chip morphology classification and tool wear prediction in CNC milling is presented in this section. The framework integrates multimodal sensor fusion, deep feature extraction, temporal sequence modelling, and Transformer-based self-attention to simultaneously classify chip morphology and estimate progressive tool wear.

The proposed framework consists of five major stages:

1. Multimodal sensor acquisition;

2. Signal preprocessing and feature extraction;

3. Deep multimodal feature learning;

4. Transformer-based temporal attention modeling;

5. Chip classification and tool wear prediction.

The entire process of the suggested framework is shown in Figure 1.

To record the dynamics of the milling process under various cutting parameters, several sensors with different characteristics were used during the CNC milling process. The acquired sensing channels are as follows:

$\bullet$ Vibration signals;

$\bullet$ AE signals;

$\bullet$ Cutting force signals;

$\bullet$ Spindle motor current;

$\bullet$ Temperature measurements.

These sensor streams are all indicative of the stability of the machine, the behavior of the chips, the thermal behavior, and the degradation of the tool.

Let the acquired sensor data be represented as follows:

where:

$\bullet$ $x_v$\(\to\)vibration signal;

$\bullet$ $x_a$\(\to\)acoustic signal;

$\bullet$ $x_f$\(\to\)force signal;

$\bullet$ $x_c$\(\to\)current signal;

$\bullet$ $x_t$\(\to\)temperature signal.

The entire dataset includes the class of the morphology of the chips and the tool wear data obtained at different spindle speeds, feed rates, and lubrication conditions.

The raw machining signals have noise, transients, and inconsistencies in the sensor. As a result, preprocessing was needed before deep learning analysis.

The preprocessing pipeline consists of the following:

$\bullet$ Low-pass filtering;

$\bullet$ Signal normalization;

$\bullet$ Window segmentation;

$\bullet$ Statistical smoothing.

The data from each sensor were normalized with min–max normalization:

where:

$\bullet$ $x$ is the original sensor value;

$\bullet$ $x_{\min}$ and $x_{\max}$ are the minimum and maximum values, respectively, of the signal.

The normalized signals were divided into fixed-length windows for modeling the temporal sequence.

To effectively capture the machining behavior, features were extracted in both the time domain and the frequency domain from each sensor modality.

The extracted statistical features include the following:

$\bullet$ Mean;

$\bullet$ Root mean square (RMS);

$\bullet$ Kurtosis;

$\bullet$ Skewness;

$\bullet$ Variance;

$\bullet$ Peak amplitude;

$\bullet$ Spectral energy.

The RMS feature is computed as follows:

where:

$\bullet$ $N$ denotes the number of signal samples;

$\bullet$ $x_i$ represents the sensor amplitude of sample $i$.

All the features extracted from all the sensor channels were merged into a single feature representation.

Complementary information from a set of heterogeneous sensor streams was fused by feature fusion.

The combined feature vector is given as follows:

where:

$\bullet$ $F_v$\(\to\)vibration features;

$\bullet$ $F_a$\(\to\)acoustic features;

$\bullet$ $F_f$\(\to\)force features;

$\bullet$ $F_c$\(\to\)current features;

$\bullet$ $F_t$\(\to\)temperature features;

$\bullet$ $\oplus$ denotes feature concatenation.

The fusion strategy enhances the robustness and discriminative power of the learning framework.

a) CNN-based Spatial Feature Extraction

Local spatial patterns in sensor sequences were extracted by using a one-dimensional CNN.

The convolution operation is defined as follows:

where:

$\bullet$ $w_k$ denotes the convolution kernel weights;

$\bullet$ $b$ is the bias term;

$\bullet$ $K$ represents the kernel size.

The CNN layers learn the local machining patterns related to chip breaking and tool wear.

b) BiLSTM Temporal Modeling

The sequential dependency of the machining signals was modeled using BiLSTM layers.

The hidden state update is given by:

where:

$\bullet$ $x_t$ denotes the input sequence;

$\bullet$ $h_t$ represents the hidden state;

$\bullet$ $W_h$ and $W_x$ are trainable weights.

BiLSTM models both forward and backward propagation of time in machining dynamics.

A multihead self-attention transformer was incorporated to capture long-range relationships between multimodal sensor features.

The self-attention operation is defined as follows:

where:

$\bullet$ $Q$\(\to\)query matrix;

$\bullet$ $K$\(\to\)key matrix;

$\bullet$ $V$\(\to\)value matrix;

$\bullet$ $d_k$\(\to\)key dimensionality.

The transformer module allows for adaptive weighting of the important features of the sensors and enhances feature interpretability.

The features of the optimized transformers were transmitted to a softmax classifier to identify the morphology of the chips.

The softmax function is given as follows:

where:

$\bullet$ $z_i$ represents class logits;

$\bullet$ $C$ denotes the number of chip categories.

This model categorizes the form of the chips into:

$\bullet$ Continuous chips;

$\bullet$ Segmented chips;

$\bullet$ Discontinuous chips;

$\bullet$ Serrated chips.

To estimate flank wear, a regression head was used with fused transformer features.

The loss function used for regression optimization is as follows:

where:

$\bullet$ $y_i$ is the actual tool wear;

$\bullet$ $\widehat{y}_i$ is the predicted tool wear.

The model is used to simulate progressive tool degradation during machining.

Classification and regression measures were used to assess the framework performance.

Classification accuracy:

Root mean square error (RMSE):

Coefficient of determination:

where, $\bar{y}$ represents the average tool wear. The proposed framework was also evaluated against existing machine learning and deep learning approaches such as RF, XGBoost, CNN-LSTM, and standalone transformer networks.

The proposed intelligent MM-DL framework was tested and validated for chip classification and tool wear prediction by using different sets of CNC milling parameters. The effectiveness of multimodal sensor fusion, transformer-based feature learning, robustness across machining environments, and the contribution of each architectural component were experimentally validated. The framework was trained with synchronized vibration, AE, force, current, and temperature sensor data from the CNC milling processes.

The results of the classification accuracy with different sensor modality configurations are shown in Figure 2. Compared with the conventional Transformer and CNN-LSTM architectures, the proposed MM-DL framework has consistently superior performance in all configurations. As a consequence, the prediction accuracy was lower for single-sensor inputs such as vibration, acoustic, or temperature signals, as the representation of machining dynamics was limited. The individual modalities showed relatively good performance with respect to force and vibration signals because of their direct connection with cutting mechanics and chip formation.

The availability of several sensing channels improved the classification capability. The combined vibration and AE (Vib+AE) configuration yielded an accuracy of approximately 93.4%, and combining the vibration and force (Vib+Force) signals further increased the accuracy to 94.1%. The best result was achieved by using the proposed framework with all the sensory modalities used together, which yielded a classification accuracy of $\sim$97.2%.

These results show that sensor fusion can be used to obtain complementary information about cutting stability, chip segmentation behavior, and tool–workpiece interactions. The transformer attention mechanism successfully learned the interdependencies between heterogeneous types of sensor streams, improving the feature representation and classification reliability.

The proposed tool wear estimation framework is presented as a regression model, which is shown in Figure 3. The tool wear values predicted are similar to the experimental results and are strongly correlated with the experimental observations. The prediction points are clustered around an ideal regression line, indicating low prediction error and high model consistency. The proposed MM-DL model outperforms the baseline machine learning techniques. The actual and predicted tool wear values are well aligned, suggesting the ability of the transformer-based fusion framework to learn the nonlinear relationships between the responses from the sensors and progressive tool degradation. In intelligent machining systems, tool wear prediction is critically important since the role that tool wear plays directly affects the dimensional accuracy, surface integrity, cutting force fluctuations, and production costs. The results show that the proposed framework has the potential to effectively facilitate predictive maintenance and real-time tool condition monitoring in CNC milling applications.

The confusion matrix shown in Figure 4 is used to assess the classification performance of the proposed model with the four different categories of chip morphology: continuous, segmented, discontinuous, and serrated chips. The classification accuracy was high for all the chip categories, with most of the samples along the diagonal elements of the confusion matrix. Continuous and serrated chip classes showed the best prediction accuracy since their signal characteristics have unique frequency and force-domain patterns. Some misclassification was experienced between segmented and discontinuous chips because under unstable cutting conditions, the transient response of the cutting process is partly similar for both types of chips. The overall classification performance, however, remained quite high and reflected a high degree of generalization of the proposed framework. The confusion matrix also confirms the ability of the transformer attention mechanism to learn discriminative temporal and spatial information from multimodal sensor inputs. Accurate recognition of chip morphology is especially critical for automated quality assessment and process optimization of machining.

To check its applicability in industry, the proposed framework was validated under various machining conditions, such as low-speed/high-feed, high-speed/low-feed, wet, and dry milling conditions. The robustness analysis presented in Figure 5 proves that the MM-DL framework exhibited a stable prediction accuracy under all operating conditions. While the prediction accuracy of the conventional transformer and XGBoost models significantly decreased under extreme machining conditions such as high-speed dry milling, the prediction accuracy of the proposed model remained above 94%, which is the benchmark for advanced artificial intelligence models. The multimodal feature fusion strategy is key to the robustness of the proposed framework, as it makes it possible to maintain reliable information when one of the sensing modalities is noisy or unstable. The higher prediction accuracy obtained under wet milling conditions was due to the lower thermal fluctuation and smooth-chip evacuation. On the other hand, the dry-milling process produced high variations in the signal as a result of high friction and thermal instability. However, the proposed architecture remained capable of maintaining its performance in both environments. The results all demonstrate the good adaptability and generalizability of the framework for use in practical intelligent machining applications.

To examine the contribution of individual elements of the proposed structure to the overall performance of the proposed framework, an ablation study was performed. These comparative results are shown in Figure 6. The CNN-only configuration had approximately 89% prediction accuracy for the machining signal, which means that convolutional feature extraction is not enough to capture the long-range temporal dependency in the machining signal. The model could be used to accurately classify sequential relationships in sensor data to achieve nearly 92% accuracy when LSTM layers were used. The stand-alone Transformer architecture also achieved the highest accuracy of $\sim$95%, highlighting the ability of self-attention mechanisms to learn temporal features. The overall proposed MM-DL framework achieved the highest prediction accuracy of approximately 98%, indicating that convolutional feature extraction, sequential modeling, transformer attention, and multimodal fusion significantly increase accuracy. The results of the ablation show that the individual modules indeed improve the predictions, thus supporting the architectural design of the proposed framework.

The experimental results show that the proposed Intelligent Multi-Modal Learning Architecture can solve the two most important problems in the field of machine manufacturing, that is, tool wear estimation and chip morphology classification. Compared with traditional deep learning and machine learning models, the proposed framework exhibited better prediction performance, robustness to different cutting conditions, and multimodal sensor fusion to enhance the generalization capability. By incorporating transformer-based attention mechanisms, complex relationships between different streams of heterogeneous sensors were efficiently learned. In addition, the multimodal fusion strategy was found to enhance the stability of the classification results and to be less sensitive to operational variations and noise. The proposed framework is thus a potential solution for intelligent manufacturing systems, predictive maintenance, adaptive process control, and Industry 4.0-based smart machining environments.

In this study, an intelligent multimodal learning architecture for chip formation classification and tool wear prediction in CNC milling operations was presented. The proposed framework aimed to address the drawbacks of traditional single-sensor and handcrafted-feature-based monitoring approaches by leveraging heterogeneous sensor signals, such as vibration, AE, cutting force, spindle current, and temperature measurements. The hybrid architecture was constructed with a CNN for spatial features, BiLSTM for temporal sequences, and a transformer for self-attention learning. This allowed both the local pattern of the machining signal and long-range dependencies in time in terms of chip formation and tool degradation to be captured by the model. The proposed framework successfully accomplished 2 main tasks of machining intelligence, namely, classification of chip morphology and prediction of progressive tool wear. The experimental results revealed that compared with any single sensor modality, multimodal sensor fusion results in significantly better classification performance. The results of the tool wear prediction values were in good agreement with the actual values, indicating that the proposed model is effective for tool condition monitoring based on regression. The confusion matrix confirmed the reliable classification of the chip morphology for all the chip categories: continuous, segmented, discontinuous, and serrated. Furthermore, a robustness analysis demonstrated the stability of the prediction performance of the proposed framework with different cutting speeds, feed rates, and lubrication conditions. The contribution of each architectural component was further validated by an ablation study. The results demonstrated that the complete proposed framework outperforms the CNN-only, CNN-LSTM, and standalone transformer models, highlighting the advantages of the fusion of multimodal data and attention-based learning in terms of predictive accuracy and generalizability. The overall architecture is found to be reliable and scalable for intelligent CNC milling monitoring purposes. It can be used for predictive maintenance, adaptive machining control, tool replacement planning, and smart manufacturing systems based on the Industry 4.0 concept. This framework can be further extended in future works through real-time edge deployment, the introduction of wider industrial datasets, vision-based chip morphology analysis, and reinforcement learning-based adaptive machining parameter optimization.