Physics-Guided Transformer Learning for Microstructural Failure Prediction in High-Entropy Alloys

High-entropy alloys are a unique group of advanced engineering materials that are known for their outstanding mechanical properties, thermal stability, corrosion resistance, and structural reliability under extreme operating conditions. These alloys are unique with multiple primary elements in near-equiatomic proportions, which give them unique microstructural behavior and improved mechanical properties [1]. Owing to these properties, high-entropy alloys have been the focus of large-scale research in various sectors, including the aerospace industry, energy, the automotive industry, and nuclear power. The prediction of the microstructural failure behavior of high-entropy alloys is still a significant challenge owing to the complicated interactions among the alloy composition, phase distribution, grain morphology, and thermodynamic stability. Traditional experimental methods for characterizing the failure mechanisms of high-entropy alloys are not only costly but also time-consuming and often lack the ability to cover a wide design space of the composition [2]. Moreover, traditional computational methods involve many simulations and fail to accurately describe the nonlinear relationships between microstructures and properties.

In recent years, machine learning and deep learning have emerged as promising approaches for accelerating material discovery and mechanical property prediction. Numerous studies have used neural networks, convolutional architectures, and ensemble learning techniques to predict the strength of high-entropy alloys, the phases that form, and the deformation properties [3]. However, the majority of current methods are purely data-driven learning methods and do not always have physical interpretability and generalizability, especially in predicting failure-related properties under different microstructural conditions. Transformers have recently been the focus of deep learning models because of their ability to model the long-range dependency of features by means of self-attention. Transformers are widely used architectures in natural language processing, and they have shown good performance for some scientific learning tasks, such as materials informatics and microstructural analysis [4]. However, even purely transformer-based models can make physically inconsistent predictions if domain-specific material constraints are not included in the learning process.

To overcome the aforementioned challenges, this study aims to develop a physics-guided transformer learning framework for microstructural failure prediction in high-entropy alloys. The suggested framework combines the physics-informed feature extraction process with the attention learning process of the transformer model to establish physically meaningful associations between the alloy composition, microstructural descriptors, and mechanical failure behavior. To increase the reliability of the predictions and their interpretability, important physical descriptors (mixing entropy, atomic size mismatch, phase stability, and valence electron concentration) are integrated into the learning process.

The proposed framework is able to predict many mechanical properties and failure properties, such as the ultimate tensile strength, failure strain, stress–strain response, and failure probability estimation. Moreover, attention-based feature analysis is used to determine the most influential microstructural parameters affecting the failure behavior of the high-entropy alloy. The framework is also validated through extensive analyses of performance, stress–strain comparisons, and ablation studies.

This work makes the following major contributions:

$\bullet$ Development of a physics-guided transformer approach for failure prediction in high-entropy alloys;

$\bullet$ Coupling of physically meaningful thermodynamic and microstructural descriptors in transformer learning;

$\bullet$ Precise prediction of tensile properties, failure strain, and stress–strain curves;

$\bullet$ Attention-based interpretability analysis for critical failure-related microstructural feature identification;

$\bullet$ Superior performance of the model demonstrated through a thorough assessment, compared with traditional machine learning and deep learning models.

The results obtained prove the effectiveness and interpretability of the proposed framework in predicting the failure of microstructures and designing materials on the basis of data with a high-entropy alloy material.

In recent years, research on high-entropy alloys has focused on understanding the complex composition–microstructure–mechanical failure behavior relationships. Because of the outstanding mechanical strength, thermal stability, and corrosion resistance of high-entropy alloys, computational and/or data-driven methods for predicting material properties and optimizing alloy design have attracted increasing interest [1], [2]. Several studies have explored the use of machine learning methods for predicting material properties. In a seminal study, Tao et al. [3] reported that machine learning frameworks can discover materials much faster by learning a nonlinear relationship between compositional descriptors and the properties of materials. Similarly, Jiang et al. [4] presented a transformer-based model for material property prediction and achieved better feature representation performance than traditional neural networks. Recently, Pokharel et al. [5] proposed physics-informed learning strategies for modeling microstructure-sensitive materials and demonstrated that physical constraints enhance prediction consistency and interpretability.

Other classical machine learning approaches, such as random forests, support vector regression, and extreme gradient boosting, are also commonly used to predict the mechanical properties of high-entropy alloys. Wen et al. [6] created predictive models with the ensemble-learning method to determine the tensile strength and phase stability of high-entropy alloys. These methods were able to predict the materials moderately well but failed to capture the complex long-range dependencies among the microstructural features. Recently, deep learning techniques have demonstrated greater capability than conventional machine-learning approaches in addressing complex materials informatics tasks. Lu et al. [7] used a convolutional neural network for microstructure-property mapping and reported that the prediction performance of deformation-related characteristics increased. Methods based on convolutional neural networks, however, generally rely on local feature extraction and are unable to capture global interactions among material descriptors.

To predict material properties, recurrent neural networks and long short-term memory models have been investigated as well. In this regard, Shang et al. [8] presented a long short-term memory-based approach to predict the thermomechanical properties of alloy systems and demonstrated enhanced sequential feature learning capability. However, long short-term memory models suffer from handling large-scale heterogeneous material datasets because of gradient degradation and limited long-range attention learning. Owing to their self-attention mechanisms, transformer architectures have proven to be very effective alternatives for scientific learning tasks. Rodríguez et al. [9] proved that transformer models can be used to elucidate the complex relationships between the compositional and thermodynamic properties of advanced materials. The work focused on the need for feature interaction modeling in complex alloy systems. In addition to deep learning architectures, many researchers have stressed the influence of physical knowledge in machine learning architectures. The physics-informed neural network has been a focus of research because it embeds governing physics in the model optimization process. Karniadakis et al. [10] demonstrated that physics-informed learning enhances the generalization ability and decreases the number of physically inconsistent predictions for scientific applications. Most of the physics-informed neural network-based methods, however, are developed mainly for the learning of partial differential equations and the modeling of fluid dynamics in high-entropy alloys, without considering failure prediction in microstructures.

The prediction of failure mechanisms at the microstructural scale is among the most difficult tasks in the field of materials science because of the complex interactions among numerous interrelated parameters, including grain size, phase morphology, dislocation behavior, and thermodynamic stability. Multiple studies have attempted to predict the propagation of cracks and deformation mechanisms on the basis of statistical and finite element methods [11]. These approaches, however, are rather costly when applied to large compositional design spaces and are based on extensive simulations. Recently, explainability approaches based on attention have been employed to enhance the interpretability of materials informatics. Duan et al. [12] used attention-based models to extract critical microstructural descriptors that affect material performance. It was reported that a physically interpretable understanding of the behavior of complex materials was obtained through attention-based learning.

While significant success has been made in machine learning-aided material design, several challenges remain in previous works:

$\bullet$ A lack of physically guided learning;

$\bullet$ Insufficient interpretability;

$\bullet$ A lack of failure prediction ability;

$\bullet$ A lack of sufficient ability to capture the interactions of microstructures globally.

To overcome these issues, the current research introduces a physics-guided transformer learning framework for microstructural failure prediction in high-entropy alloys. In contrast to traditional data-driven models, the proposed framework combines physics-inspired descriptors such as mixing entropy, atomic size mismatch, and valence electron concentration with attention learning through a transformer. This results in enhanced predictive power, understanding of the physics, and interpretability in the high-entropy alloy failure analysis. The proposed framework is further enhanced by the inclusion of attention-based feature analysis, stress–strain prediction, and failure probability estimation to offer an all-in-one approach for intelligent microstructural failure prediction in advanced alloy systems.

In this research, a physics-guided transformer learning framework is developed to predict the failure behavior of microstructures in high-entropy alloys. The proposed framework combines physical descriptors developed from materials sciences with a transformer-based deep learning method to establish accurate relationships among the alloy composition, microstructural characteristics, and mechanical failure properties. The proposed framework also introduces physically relevant constraints related to phase stability, lattice distortion, and thermodynamics, which are not used in traditional machine learning models that rely solely on statistical correlations. This helps enhance the reliability of predictions and the generalization of the systems used for complex high-entropy alloys. The overall flow of the proposed framework is shown in Figure 1.

The overall methodology comprises the following seven stages:

Stage 1: Acquisition and preprocessing of high-entropy alloy data;

Stage 2: Physics-guided feature extraction;

Stage 3: Transformer-based representation learning;

Stage 4: Analysis of feature interactions with an attention mechanism;

Stage 5: Mechanical property prediction;

Stage 6: Failure probability estimation;

Stage 7: Optimum design and testing of the models.

The high-entropy alloy dataset comprises experimentally reported alloy systems exhibiting face-centered cubic, body-centered cubic, dual-phase face-centered cubic + body-centered cubic, intermetallic, and amorphous structures. The data present both compositional and mechanical properties, including grain size, hardness, density, ultimate tensile strength, and failure strain. All the features are preprocessed using min–max normalization before training:

where, $X$ represents the original feature value, and $X_{\min}$ and $X_{\max}$ denote the minimum and maximum feature values, respectively. Normalization makes the training more stable and the convergence of the model faster.

The framework also includes thermodynamic and microstructural descriptors that are widely used in the analysis of high-entropy alloys to increase its physical interpretability. Mixing entropy is expressed as follows:

where, $R$ is the gas constant, and $c _i$ represents the atomic concentration.

The equation for atomic size mismatch is as follows:

where, $r _i$ denotes the atomic radius, and $\bar{r}$ represents the average atomic radius.

Valence electron concentrations are expressed as follows:

The descriptors aid in the representation of phase stability and strengthening behaviors linked to the failure mechanisms of high-entropy alloys.

The architecture of the proposed model is based on a transformer encoder network that can model the nonlinear relationships between microstructural features and mechanical properties. The input feature vector is given as follows:

The transformer attention mechanism is defined as follows:

where, $Q$ is the query matrix, $K$ is the key matrix, $V$ is the value matrix, and $d_k$ is the dimensional scaling factor. The model captures the intricate relationships among the phase structure, grain morphology, and mechanical behavior by means of a multihead attention mechanism.

The overall loss function ($L_\text{total}$) is a combination of prediction loss ($L_\text{data}$) and physics-based regularization ($\lambda L_\text{physics}$, $\lambda$ is the weight coefficient):

The prediction loss is calculated as follows:

where, $y_i$ denotes the actual mechanical property, $N$ denotes the number of samples, and $\hat{y}_i$ denotes the predicted property. The physics-guided term forces the model toward physically consistent material behavior.

The proposed scheme foresees the ultimate tensile strength, failure strain, stress–strain response, and failure probability. The failure probability is estimated by applying sigmoid activation:

where, $P_f$ represents the failure probability, and $z$ is the latent feature representation. A higher failure probability indicates a greater risk of microstructural failure.

The framework is trained using the Adam optimizer:

where, $\theta$ is the model parameter, $\eta$ is the learning rate, $\hat{m}_t$ is the bias-corrected first moment estimate, $\hat{v}_t$ is the bias-corrected second raw moment estimate, $\epsilon$ is the smoothing term, and $t$ is the time step. The dataset is divided into training (80%) and testing (20%) sets. The performance evaluation metrics include the coefficient of determination ($R^2$), the mean absolute error, and the root mean square error, which are defined as follows:

Accurate and interpretable microstructural failure prediction in high-entropy alloys is achieved by combining physics-guided learning with transformer attention.

The distribution of high-entropy alloy samples is provided in Figure 2a and is based on the different phases of the microstructure. The dataset contains face-centered cubic, body-centered cubic, face-centered cubic+body-centered cubic, intermetallic, and amorphous phase categories. These include the largest number of face-centered cubic-based high-entropy alloys, body-centered cubic, and mixed face-centered cubic + body-centered cubic phases. The significance of this distribution lies in the fact that the phase constitution significantly affects the mechanical strength, ductility, and failure behavior of high-entropy alloys. Multiple-phase classes enable the proposed physics-guided transformer framework to develop various microstructure–property relationships. The phasewise distribution of data enables effective training for microstructural failure prediction, as failure behavior in high-entropy alloys is dependent on phase stability, grain structure, and microstructural heterogeneity.

The loss curves for training and validation of the proposed physics-guided transformer model are illustrated in Figure 2b. The training and validation losses steadily decrease as training progresses, indicating stable learning and effective convergence. The training and validation losses are closely correlated, which implies that the model is not heavily overfitting. It is clear from the gradual decline in validation loss that the proposed model is able to generalize well to unseen high-entropy alloy samples. This further validates the ability of a physics-informed and physics-guided integration of transformer-based feature learning and physics-based constraints to enhance the robustness of microstructural failure prediction.

The calculated and experimental ultimate tensile strength against the values are shown in Figure 2c. The predicted points are concentrated around the diagonal reference line, indicating good agreement between the measured and predicted ultimate tensile strength values. This outcome demonstrates the effectiveness of the proposed framework for determining the relationships among the alloy composition, microstructural descriptors, and tensile strength. Ultimate tensile strength prediction is crucial since tensile strength is among the most important mechanical properties for the reliability and failure resistance of high-entropy alloy materials.

The predicted and actual values of failure strain are plotted in Figure 2d. The predicted failure strain values closely agree with the actual values across the entire strain range, indicating the model's strong predictive performance in characterizing ductility and deformation behavior. These results directly assist the main goal of the study, namely, the prediction of microstructural failure. Failure strain characterizes the deformation capacity of a material prior to fracture and is therefore an important parameter for assessing the mechanical safety and application suitability of high-entropy alloys.

To assess the performance of various machine learning and deep learning models, the random forest, extreme gradient boosting, convolutional neural network, long short-term memory, transformer, and proposed physics-guided transformer models are compared in terms of predictive performance, as shown in Figure 3a. Compared with all the baseline models, the proposed model has the maximum $R^2$ value. The improved performance relative to traditional machine learning models, such as random forest and extreme gradient boosting, suggests that deep representation learning may more effectively capture the nonlinear relationships between microstructural characteristics and material properties of high-entropy alloys. The proposed model also outperforms the convolutional neural network, long short-term memory, and regular transformer models, highlighting the advantages of adding physics-informed learning to the transformer model.

The experimental and predicted stress–strain curves are shown in Figure 3b. The curve predicted is similar to the experimental curve in both the elastic and plastic areas. This suggests that the model under consideration is capable of mimicking the mechanical deformation response of high-entropy alloys. The good agreement between the predicted and experimental stress–strain curves validates the proposed physics-guided transformer framework for predicting not only the properties at specific points but also the overall deformation behavior. This can increase the model's applicability for failure analysis and mechanical reliability assessment.

The microstructural failure probability heatmap developed by the proposed model is shown in Figure 3c. The intensity of the areas in different colors represents the probability of failure, whereas the lower intensity areas represent relatively stable areas of the microstructure. The heatmap helps to visualize failure-prone areas related to microstructural features and serves as visual evidence of failure. Understanding how failure behavior is affected by the phase distribution, grain boundary characteristics, and local structural variations is crucial. This improves the interpretability of the proposed framework and aids in the use of this framework for microstructure-guided material design.

The results of the transformer feature attention analysis are shown in Figure 3d. The order of attention that is given to grain size is phase fraction, valence electron concentration, entropy, hardness, density, and heat-treatment temperature. This means that the model gives greater weight to microstructural and compositional features, which are highly correlated with mechanical failure. The attention-based analysis shows that the model is not a black-box predictor. Rather, it involves the identification of physically meaningful variables that affect the failure behavior of the high-entropy alloy. The grain size and phase fraction are particularly relevant since they directly affect the dislocation mobility, phase stability, strengthening mechanisms, and fracture resistance.

The ablation study of the proposed framework is presented in Figure 4. The results are presented and compared with those of the transformer-only, transformer with attention, physics-guided transformer, and the complete proposed framework. The complete proposed framework achieves the highest $R^2$ and failure prediction accuracy while yielding the lowest mean absolute error for ultimate tensile strength prediction. In other words, each of these components is responsible for the overall model performance. The standard transformer provides better sequence-based feature learning, the attention mechanism provides better feature relevance extraction, and the physics-guided part provides better consistency in predicting material behavior. The complete framework offers the best compromise in terms of accuracy, interpretability, and physical reliability.

The findings highlight the effectiveness of the proposed physics-guided transformer learning framework for failure prediction for materials such as high-entropy alloys. Through heatmaps and attention visualization, the model can provide interpretable microstructural failure analysis with high-accuracy prediction of key mechanical properties such as the ultimate tensile strength and failure strain of components. The proposed model outperforms traditional machine learning and deep learning models by combining transformer-based attention learning with physics-guided constraints. The stress‒strain comparison curve is also provided to confirm the mechanical deformation behavior of the model; thus, it is applicable to material reliability analysis. The proposed framework is a promising data-driven and physics-informed solution for both high-entropy alloy failure prediction and microstructure–property mapping, as well as the intelligent design of alloys.

In this research, a physics-guided transformer learning framework was proposed to predict the failure of microstructural components in high-entropy alloys. The framework was created to address the drawbacks of traditional data-driven models, introducing both physically meaningful descriptors and attention learning with transformers. By combining the mixing entropy, atomic size mismatch, valence electron concentration, phase stability descriptors, and microstructural features, the model was able to effectively address the complex relationships among the composition, microstructure, and mechanical failure behavior of high-entropy alloys. The experimental results revealed that the proposed framework was able to accurately predict the ultimate tensile strength and failure strain, and the predicted values were close to the actual values. A comparison of the stress–strain curves also revealed that the model was able to capture the deformation behavior of high-entropy alloys. Furthermore, the microstructural failure probability heatmap revealed regions that were likely to fail, and the transformer attention analysis revealed that the grain size, phase fraction, valence electron concentration, entropy, and hardness were influencing parameters for failure prediction. The proposed physics-guided transformer demonstrated better prediction accuracy than the other models, such as the random forest, extreme gradient boosting, the convolutional neural network, long short-term memory, and standard transformer models. Abnormalities in physics-guided learning and attention-based feature modeling significantly increased the accuracy, interpretability, and physical consistency, as the ablation study demonstrated. In general, predicting the mechanical properties of high-entropy alloys and analyzing the failure mechanisms at the microstructural level are reliable and interpretable methods. It can be applicable to intelligent alloy design, failure-risk assessment, and accelerated material discovery. This framework can be extended in the future by expanding the experimental data and introducing image-based microstructure data from microscopy, finite element simulation data, and experimental data under different loading and thermal conditions.