Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models

Adem Avcı; Nurettin Acır

doi:10.38088/jise.1206920

EN

Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models

Abstract

Prognostics and Health Management occupy an important place in modern industrial maintenance to increase the reliability of systems. Determining the Remaining Useful Life of the system or its parts is vital accurately to maintaining critical parts of the system and successful prognostics and health management. This study proposes a data-based Remaining Useful Life prediction method with a network consisting of a cascade-connected Self-Attention and Residual Network layer. The network is fed by multiple sensor signals to monitor the aero-engines. The proposed model contains four main parts: The Gaussian Noise Layer, the Self-Attention Layer, the Residual Network Layer, and the layer to estimate Remaining Useful Life. The model is created to be more robust and susceptible to noise using the Gaussian Noise Layer. The Self-Attention Layer focuses on crucial points through time. The Residual Network Layer uses feature extraction and makes the model more profound help of the skip connection. Finally, the Remaining Useful Life estimation is made using highly correlated features obtained from the fully connected layer and the output layer. In addition, a new loss function has been offered, similar to the evaluation metrics in the literature. With the proposed model and loss function, 11.017 and 12.629 in root mean square error, 157.19 and 218.6 in score function are obtained in the FD001 and FD003, respectively. The superior performance of these results on the C-MAPSS dataset is demonstrated by comparing the other state-of-the-art methods in the literature.

Keywords

References

Xu, J., Wang, Y., & Xu, L. (2013). PHM-oriented integrated fusion prognostics for aircraft engines based on sensor data. IEEE Sensors Journal, 14(4), 1124-1132.
Nandi, S., Toliyat, H. A., & Li, X. (2005). Condition monitoring and fault diagnosis of electrical motors—A review. IEEE transactions on energy conversion, 20(4), 719-729.
Kara, A. (2021). A data-driven approach based on deep neural networks for lithium-ion battery prognostics. Neural Computing and Applications, 33(20), 13525-13538.
Park, J., & Jung, W. (2015). A systematic framework to investigate the coverage of abnormal operating procedures in nuclear power plants. Reliability Engineering & System Safety, 138, 21-30.
Hou, G., Xu, S., Zhou, N., Yang, L., & Fu, Q. (2020). Remaining useful life estimation using deep convolutional generative adversarial networks based on an autoencoder scheme. Computational Intelligence and Neuroscience, 2020.
Li, H., Zhao, W., Zhang, Y., & Zio, E. (2020). Remaining useful life prediction using multi-scale deep convolutional neural network. Applied Soft Computing, 89, 106113.
Correia, J. A., De Jesus, A. M., & Fernández‐Canteli, A. (2012). A procedure to derive probabilistic fatigue crack propagation data. International Journal of Structural Integrity.
Liao, L. (2013). Discovering prognostic features using genetic programming in remaining useful life prediction. IEEE Transactions on Industrial Electronics, 61(5), 2464-2472.

Li, N., Lei, Y., Lin, J., & Ding, S. X. (2015). An improved exponential model for predicting remaining useful life of rolling element bearings. IEEE Transactions on Industrial Electronics, 62(12), 7762-7773.
Sateesh Babu, G., Zhao, P., & Li, X. L. (2016, April). Deep convolutional neural network based regression approach for estimation of remaining useful life. In International conference on database systems for advanced applications (pp. 214-228). Springer, Cham.
Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2016). Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE transactions on neural networks and learning systems, 28(10), 2306-2318.
Li, X., Ding, Q., & Sun, J. Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering & System Safety, 172, 1-11.
Zheng, S., Ristovski, K., Farahat, A., & Gupta, C. (2017, June). Long short-term memory network for remaining useful life estimation. In 2017 IEEE international conference on prognostics and health management (ICPHM) (pp. 88-95). IEEE.
Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018). Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing, 275, 167-179.
Yu, W., Kim, I. Y., & Mechefske, C. (2019). Remaining useful life estimation using a bidirectional recurrent neural network based autoencoder scheme. Mechanical Systems and Signal Processing, 129, 764-780
Wang, J., Wen, G., Yang, S., & Liu, Y. (2018, October). Remaining useful life estimation in prognostics using deep bidirectional LSTM neural network. In 2018 Prognostics and System Health Management Conference (PHM-Chongqing) (pp. 1037-1042). IEEE.
Ruiz-Tagle Palazuelos, A., Droguett, E. L., & Pascual, R. (2020). A novel deep capsule neural network for remaining useful life estimation. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 234(1), 151-167.
Al-Dulaimi, A., Zabihi, S., Asif, A., & Mohammadi, A. (2019). A multimodal and hybrid deep neural network model for remaining useful life estimation. Computers in industry, 108, 186-196.
Al-Dulaimi, A., Zabihi, S., Asif, A., & Mohammed, A. (2020). NBLSTM: Noisy and hybrid convolutional neural network and BLSTM-Based deep architecture for remaining useful life estimation. Journal of Computing and Information Science in Engineering, 20(2), 021012.
Li, J., Li, X., & He, D. (2019). A directed acyclic graph network combined with CNN and LSTM for remaining useful life prediction. IEEE Access, 7, 75464-75475.
Song, Y., Shi, G., Chen, L., Huang, X., & Xia, T. (2018). Remaining useful life prediction of turbofan engine using hybrid model based on autoencoder and bidirectional long short-term memory. Journal of Shanghai Jiaotong University (Science), 23(1), 85-94.
Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Yan, R., & Li, X. (2021). Attention-based sequence to sequence model for machine remaining useful life prediction. Neurocomputing, 466, 58-68.
Liu, L., Song, X., & Zhou, Z. (2022). Aircraft engine remaining useful life estimation via a double attention-based data-driven architecture. Reliability Engineering & System Safety, 221, 108330.
Tan, W. M., & Teo, T. H. (2021). Remaining useful life prediction using temporal convolution with attention. Ai, 2(1), 48-70.
Xia, J., Feng, Y., Lu, C., Fei, C., & Xue, X. (2021). LSTM-based multi-layer self-attention method for remaining useful life estimation of mechanical systems. Engineering Failure Analysis, 125, 105385.
Zhao, H., Jia, J., & Koltun, V. (2020). Exploring self-attention for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10076-10085).
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008, October). Damage propagation modeling for aircraft engine run-to-failure simulation. In 2008 international conference on prognostics and health management (pp. 1-9). IEEE.
Xia, J., Feng, Y., Lu, C., Fei, C., & Xue, X. (2021). LSTM-based multi-layer self-attention method for remaining useful life estimation of mechanical systems. Engineering Failure Analysis, 125, 105385.
Heimes, F. O. (2008, October). Recurrent neural networks for remaining useful life estimation. In 2008 international conference on prognostics and health management (pp. 1-6). IEEE.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Yang, H., Zhao, F., Jiang, G., Sun, Z., & Mei, X. (2019). A novel deep learning approach for machinery prognostics based on time windows. Applied Sciences, 9(22), 4813.
Song, J. W., Park, Y. I., Hong, J. J., Kim, S. G., & Kang, S. J. (2021, May). Attention-based bidirectional LSTM-CNN model for remaining useful life estimation. In 2021 IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 1-5). IEEE.
Jin, R., Chen, Z., Wu, K., Wu, M., Li, X., & Yan, R. (2022). Bi-LSTM-Based Two-Stream Network for Machine Remaining Useful Life Prediction. IEEE Transactions on Instrumentation and Measurement, 71, 1-10.
Zhang, J., Jiang, Y., Wu, S., Li, X., Luo, H., & Yin, S. (2022). Prediction of remaining useful life based on bidirectional gated recurrent unit with temporal self-attention mechanism. Reliability Engineering & System Safety, 221, 108297.

Details

Primary Language

English

Subjects

Software Engineering , Electrical Engineering

Journal Section

Research Article

Authors

Adem Avcı ^*
0000-0002-1529-8765
Türkiye

Nurettin Acır
0009-0001-4796-9092
Türkiye

Early Pub Date

June 19, 2023

Publication Date

June 21, 2023

Submission Date

November 18, 2022

Acceptance Date

February 23, 2023

Published in Issue

Year 2023 Volume: 7 Number: 1

DOI

https://doi.org/10.38088/jise.1206920

IZ

https://izlik.org/JA63PE82UW

Cite

RIS / Bibtex

APA

Avcı, A., & Acır, N. (2023). Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. Journal of Innovative Science and Engineering, 7(1), 88-105. https://doi.org/10.38088/jise.1206920

AMA

1.Avcı A, Acır N. Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. JISE. 2023;7(1):88-105. doi:10.38088/jise.1206920

Chicago

Avcı, Adem, and Nurettin Acır. 2023. “Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models”. Journal of Innovative Science and Engineering 7 (1): 88-105. https://doi.org/10.38088/jise.1206920.

EndNote

Avcı A, Acır N (June 1, 2023) Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. Journal of Innovative Science and Engineering 7 1 88–105.

IEEE

[1]A. Avcı and N. Acır, “Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models”, JISE, vol. 7, no. 1, pp. 88–105, June 2023, doi: 10.38088/jise.1206920.

ISNAD

Avcı, Adem - Acır, Nurettin. “Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models”. Journal of Innovative Science and Engineering 7/1 (June 1, 2023): 88-105. https://doi.org/10.38088/jise.1206920.

JAMA

1.Avcı A, Acır N. Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. JISE. 2023;7:88–105.

MLA

Avcı, Adem, and Nurettin Acır. “Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models”. Journal of Innovative Science and Engineering, vol. 7, no. 1, June 2023, pp. 88-105, doi:10.38088/jise.1206920.

Vancouver

1.Adem Avcı, Nurettin Acır. Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. JISE. 2023 Jun. 1;7(1):88-105. doi:10.38088/jise.1206920

Öz

Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models

Abstract

Keywords

References

Details

Primary Language

Subjects

Journal Section

Authors

Early Pub Date

Publication Date

Submission Date

Acceptance Date

Published in Issue

DOI

IZ

Cite