Research Article
BibTex RIS Cite

Year 2023, Volume: 7 Issue: 1, 88 - 105, 21.06.2023

Adem Avcı , Nurettin Acır

https://doi.org/10.38088/jise.1206920

Abstract

References

  • Xu, J., Wang, Y., & Xu, L. (2013). PHM-oriented integrated fusion prognostics for aircraft engines based on sensor data. IEEE Sensors Journal, 14(4), 1124-1132.
  • Nandi, S., Toliyat, H. A., & Li, X. (2005). Condition monitoring and fault diagnosis of electrical motors—A review. IEEE transactions on energy conversion, 20(4), 719-729.
  • Kara, A. (2021). A data-driven approach based on deep neural networks for lithium-ion battery prognostics. Neural Computing and Applications, 33(20), 13525-13538.
  • Park, J., & Jung, W. (2015). A systematic framework to investigate the coverage of abnormal operating procedures in nuclear power plants. Reliability Engineering & System Safety, 138, 21-30.
  • Hou, G., Xu, S., Zhou, N., Yang, L., & Fu, Q. (2020). Remaining useful life estimation using deep convolutional generative adversarial networks based on an autoencoder scheme. Computational Intelligence and Neuroscience, 2020.
  • Li, H., Zhao, W., Zhang, Y., & Zio, E. (2020). Remaining useful life prediction using multi-scale deep convolutional neural network. Applied Soft Computing, 89, 106113.
  • Correia, J. A., De Jesus, A. M., & Fernández‐Canteli, A. (2012). A procedure to derive probabilistic fatigue crack propagation data. International Journal of Structural Integrity.
  • Liao, L. (2013). Discovering prognostic features using genetic programming in remaining useful life prediction. IEEE Transactions on Industrial Electronics, 61(5), 2464-2472.
  • Li, N., Lei, Y., Lin, J., & Ding, S. X. (2015). An improved exponential model for predicting remaining useful life of rolling element bearings. IEEE Transactions on Industrial Electronics, 62(12), 7762-7773.
  • Sateesh Babu, G., Zhao, P., & Li, X. L. (2016, April). Deep convolutional neural network based regression approach for estimation of remaining useful life. In International conference on database systems for advanced applications (pp. 214-228). Springer, Cham.
  • Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2016). Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE transactions on neural networks and learning systems, 28(10), 2306-2318.
  • Li, X., Ding, Q., & Sun, J. Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering & System Safety, 172, 1-11.
  • Zheng, S., Ristovski, K., Farahat, A., & Gupta, C. (2017, June). Long short-term memory network for remaining useful life estimation. In 2017 IEEE international conference on prognostics and health management (ICPHM) (pp. 88-95). IEEE.
  • Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018). Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing, 275, 167-179.
  • Yu, W., Kim, I. Y., & Mechefske, C. (2019). Remaining useful life estimation using a bidirectional recurrent neural network based autoencoder scheme. Mechanical Systems and Signal Processing, 129, 764-780
  • Wang, J., Wen, G., Yang, S., & Liu, Y. (2018, October). Remaining useful life estimation in prognostics using deep bidirectional LSTM neural network. In 2018 Prognostics and System Health Management Conference (PHM-Chongqing) (pp. 1037-1042). IEEE.
  • Ruiz-Tagle Palazuelos, A., Droguett, E. L., & Pascual, R. (2020). A novel deep capsule neural network for remaining useful life estimation. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 234(1), 151-167.
  • Al-Dulaimi, A., Zabihi, S., Asif, A., & Mohammadi, A. (2019). A multimodal and hybrid deep neural network model for remaining useful life estimation. Computers in industry, 108, 186-196.
  • Al-Dulaimi, A., Zabihi, S., Asif, A., & Mohammed, A. (2020). NBLSTM: Noisy and hybrid convolutional neural network and BLSTM-Based deep architecture for remaining useful life estimation. Journal of Computing and Information Science in Engineering, 20(2), 021012.
  • Li, J., Li, X., & He, D. (2019). A directed acyclic graph network combined with CNN and LSTM for remaining useful life prediction. IEEE Access, 7, 75464-75475.
  • Song, Y., Shi, G., Chen, L., Huang, X., & Xia, T. (2018). Remaining useful life prediction of turbofan engine using hybrid model based on autoencoder and bidirectional long short-term memory. Journal of Shanghai Jiaotong University (Science), 23(1), 85-94.
  • Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Yan, R., & Li, X. (2021). Attention-based sequence to sequence model for machine remaining useful life prediction. Neurocomputing, 466, 58-68.
  • Liu, L., Song, X., & Zhou, Z. (2022). Aircraft engine remaining useful life estimation via a double attention-based data-driven architecture. Reliability Engineering & System Safety, 221, 108330.
  • Tan, W. M., & Teo, T. H. (2021). Remaining useful life prediction using temporal convolution with attention. Ai, 2(1), 48-70.
  • Xia, J., Feng, Y., Lu, C., Fei, C., & Xue, X. (2021). LSTM-based multi-layer self-attention method for remaining useful life estimation of mechanical systems. Engineering Failure Analysis, 125, 105385.
  • Zhao, H., Jia, J., & Koltun, V. (2020). Exploring self-attention for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10076-10085).
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
  • Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008, October). Damage propagation modeling for aircraft engine run-to-failure simulation. In 2008 international conference on prognostics and health management (pp. 1-9). IEEE.
  • Xia, J., Feng, Y., Lu, C., Fei, C., & Xue, X. (2021). LSTM-based multi-layer self-attention method for remaining useful life estimation of mechanical systems. Engineering Failure Analysis, 125, 105385.
  • Heimes, F. O. (2008, October). Recurrent neural networks for remaining useful life estimation. In 2008 international conference on prognostics and health management (pp. 1-6). IEEE.
  • Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  • Yang, H., Zhao, F., Jiang, G., Sun, Z., & Mei, X. (2019). A novel deep learning approach for machinery prognostics based on time windows. Applied Sciences, 9(22), 4813.
  • Song, J. W., Park, Y. I., Hong, J. J., Kim, S. G., & Kang, S. J. (2021, May). Attention-based bidirectional LSTM-CNN model for remaining useful life estimation. In 2021 IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 1-5). IEEE.
  • Jin, R., Chen, Z., Wu, K., Wu, M., Li, X., & Yan, R. (2022). Bi-LSTM-Based Two-Stream Network for Machine Remaining Useful Life Prediction. IEEE Transactions on Instrumentation and Measurement, 71, 1-10.
  • Zhang, J., Jiang, Y., Wu, S., Li, X., Luo, H., & Yin, S. (2022). Prediction of remaining useful life based on bidirectional gated recurrent unit with temporal self-attention mechanism. Reliability Engineering & System Safety, 221, 108297.

Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models

Year 2023, Volume: 7 Issue: 1, 88 - 105, 21.06.2023

Adem Avcı , Nurettin Acır

https://doi.org/10.38088/jise.1206920

Abstract

Prognostics and Health Management occupy an important place in modern industrial maintenance to increase the reliability of systems. Determining the Remaining Useful Life of the system or its parts is vital accurately to maintaining critical parts of the system and successful prognostics and health management. This study proposes a data-based Remaining Useful Life prediction method with a network consisting of a cascade-connected Self-Attention and Residual Network layer. The network is fed by multiple sensor signals to monitor the aero-engines. The proposed model contains four main parts: The Gaussian Noise Layer, the Self-Attention Layer, the Residual Network Layer, and the layer to estimate Remaining Useful Life. The model is created to be more robust and susceptible to noise using the Gaussian Noise Layer. The Self-Attention Layer focuses on crucial points through time. The Residual Network Layer uses feature extraction and makes the model more profound help of the skip connection. Finally, the Remaining Useful Life estimation is made using highly correlated features obtained from the fully connected layer and the output layer. In addition, a new loss function has been offered, similar to the evaluation metrics in the literature. With the proposed model and loss function, 11.017 and 12.629 in root mean square error, 157.19 and 218.6 in score function are obtained in the FD001 and FD003, respectively. The superior performance of these results on the C-MAPSS dataset is demonstrated by comparing the other state-of-the-art methods in the literature.

Keywords

Remaining useful life Self-attention Prognostics and health management Deep learning Residual Layer

References

  • Xu, J., Wang, Y., & Xu, L. (2013). PHM-oriented integrated fusion prognostics for aircraft engines based on sensor data. IEEE Sensors Journal, 14(4), 1124-1132.
  • Nandi, S., Toliyat, H. A., & Li, X. (2005). Condition monitoring and fault diagnosis of electrical motors—A review. IEEE transactions on energy conversion, 20(4), 719-729.
  • Kara, A. (2021). A data-driven approach based on deep neural networks for lithium-ion battery prognostics. Neural Computing and Applications, 33(20), 13525-13538.
  • Park, J., & Jung, W. (2015). A systematic framework to investigate the coverage of abnormal operating procedures in nuclear power plants. Reliability Engineering & System Safety, 138, 21-30.
  • Hou, G., Xu, S., Zhou, N., Yang, L., & Fu, Q. (2020). Remaining useful life estimation using deep convolutional generative adversarial networks based on an autoencoder scheme. Computational Intelligence and Neuroscience, 2020.
  • Li, H., Zhao, W., Zhang, Y., & Zio, E. (2020). Remaining useful life prediction using multi-scale deep convolutional neural network. Applied Soft Computing, 89, 106113.
  • Correia, J. A., De Jesus, A. M., & Fernández‐Canteli, A. (2012). A procedure to derive probabilistic fatigue crack propagation data. International Journal of Structural Integrity.
  • Liao, L. (2013). Discovering prognostic features using genetic programming in remaining useful life prediction. IEEE Transactions on Industrial Electronics, 61(5), 2464-2472.
  • Li, N., Lei, Y., Lin, J., & Ding, S. X. (2015). An improved exponential model for predicting remaining useful life of rolling element bearings. IEEE Transactions on Industrial Electronics, 62(12), 7762-7773.
  • Sateesh Babu, G., Zhao, P., & Li, X. L. (2016, April). Deep convolutional neural network based regression approach for estimation of remaining useful life. In International conference on database systems for advanced applications (pp. 214-228). Springer, Cham.
  • Zhang, C., Lim, P., Qin, A. K., & Tan, K. C. (2016). Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics. IEEE transactions on neural networks and learning systems, 28(10), 2306-2318.
  • Li, X., Ding, Q., & Sun, J. Q. (2018). Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering & System Safety, 172, 1-11.
  • Zheng, S., Ristovski, K., Farahat, A., & Gupta, C. (2017, June). Long short-term memory network for remaining useful life estimation. In 2017 IEEE international conference on prognostics and health management (ICPHM) (pp. 88-95). IEEE.
  • Wu, Y., Yuan, M., Dong, S., Lin, L., & Liu, Y. (2018). Remaining useful life estimation of engineered systems using vanilla LSTM neural networks. Neurocomputing, 275, 167-179.
  • Yu, W., Kim, I. Y., & Mechefske, C. (2019). Remaining useful life estimation using a bidirectional recurrent neural network based autoencoder scheme. Mechanical Systems and Signal Processing, 129, 764-780
  • Wang, J., Wen, G., Yang, S., & Liu, Y. (2018, October). Remaining useful life estimation in prognostics using deep bidirectional LSTM neural network. In 2018 Prognostics and System Health Management Conference (PHM-Chongqing) (pp. 1037-1042). IEEE.
  • Ruiz-Tagle Palazuelos, A., Droguett, E. L., & Pascual, R. (2020). A novel deep capsule neural network for remaining useful life estimation. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 234(1), 151-167.
  • Al-Dulaimi, A., Zabihi, S., Asif, A., & Mohammadi, A. (2019). A multimodal and hybrid deep neural network model for remaining useful life estimation. Computers in industry, 108, 186-196.
  • Al-Dulaimi, A., Zabihi, S., Asif, A., & Mohammed, A. (2020). NBLSTM: Noisy and hybrid convolutional neural network and BLSTM-Based deep architecture for remaining useful life estimation. Journal of Computing and Information Science in Engineering, 20(2), 021012.
  • Li, J., Li, X., & He, D. (2019). A directed acyclic graph network combined with CNN and LSTM for remaining useful life prediction. IEEE Access, 7, 75464-75475.
  • Song, Y., Shi, G., Chen, L., Huang, X., & Xia, T. (2018). Remaining useful life prediction of turbofan engine using hybrid model based on autoencoder and bidirectional long short-term memory. Journal of Shanghai Jiaotong University (Science), 23(1), 85-94.
  • Ragab, M., Chen, Z., Wu, M., Kwoh, C. K., Yan, R., & Li, X. (2021). Attention-based sequence to sequence model for machine remaining useful life prediction. Neurocomputing, 466, 58-68.
  • Liu, L., Song, X., & Zhou, Z. (2022). Aircraft engine remaining useful life estimation via a double attention-based data-driven architecture. Reliability Engineering & System Safety, 221, 108330.
  • Tan, W. M., & Teo, T. H. (2021). Remaining useful life prediction using temporal convolution with attention. Ai, 2(1), 48-70.
  • Xia, J., Feng, Y., Lu, C., Fei, C., & Xue, X. (2021). LSTM-based multi-layer self-attention method for remaining useful life estimation of mechanical systems. Engineering Failure Analysis, 125, 105385.
  • Zhao, H., Jia, J., & Koltun, V. (2020). Exploring self-attention for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10076-10085).
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
  • Saxena, A., Goebel, K., Simon, D., & Eklund, N. (2008, October). Damage propagation modeling for aircraft engine run-to-failure simulation. In 2008 international conference on prognostics and health management (pp. 1-9). IEEE.
  • Xia, J., Feng, Y., Lu, C., Fei, C., & Xue, X. (2021). LSTM-based multi-layer self-attention method for remaining useful life estimation of mechanical systems. Engineering Failure Analysis, 125, 105385.
  • Heimes, F. O. (2008, October). Recurrent neural networks for remaining useful life estimation. In 2008 international conference on prognostics and health management (pp. 1-6). IEEE.
  • Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  • Yang, H., Zhao, F., Jiang, G., Sun, Z., & Mei, X. (2019). A novel deep learning approach for machinery prognostics based on time windows. Applied Sciences, 9(22), 4813.
  • Song, J. W., Park, Y. I., Hong, J. J., Kim, S. G., & Kang, S. J. (2021, May). Attention-based bidirectional LSTM-CNN model for remaining useful life estimation. In 2021 IEEE International Symposium on Circuits and Systems (ISCAS) (pp. 1-5). IEEE.
  • Jin, R., Chen, Z., Wu, K., Wu, M., Li, X., & Yan, R. (2022). Bi-LSTM-Based Two-Stream Network for Machine Remaining Useful Life Prediction. IEEE Transactions on Instrumentation and Measurement, 71, 1-10.
  • Zhang, J., Jiang, Y., Wu, S., Li, X., Luo, H., & Yin, S. (2022). Prediction of remaining useful life based on bidirectional gated recurrent unit with temporal self-attention mechanism. Reliability Engineering & System Safety, 221, 108297.
There are 36 citations in total.

Details

Primary Language English
Subjects Software Engineering, Electrical Engineering
Journal Section Research Articles
Authors

Adem Avcı 0000-0002-1529-8765

Nurettin Acır 0009-0001-4796-9092

Early Pub Date June 19, 2023
Publication Date June 21, 2023
Published in Issue Year 2023Volume: 7 Issue: 1

Cite

APA Avcı, A., & Acır, N. (2023). Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. Journal of Innovative Science and Engineering, 7(1), 88-105. https://doi.org/10.38088/jise.1206920
AMA Avcı A, Acır N. Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. JISE. June 2023;7(1):88-105. doi:10.38088/jise.1206920
Chicago Avcı, Adem, and Nurettin Acır. “Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models”. Journal of Innovative Science and Engineering 7, no. 1 (June 2023): 88-105. https://doi.org/10.38088/jise.1206920.
EndNote Avcı A, Acır N (June 1, 2023) Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. Journal of Innovative Science and Engineering 7 1 88–105.
IEEE A. Avcı and N. Acır, “Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models”, JISE, vol. 7, no. 1, pp. 88–105, 2023, doi: 10.38088/jise.1206920.
ISNAD Avcı, Adem - Acır, Nurettin. “Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models”. Journal of Innovative Science and Engineering 7/1 (June 2023), 88-105. https://doi.org/10.38088/jise.1206920.
JAMA Avcı A, Acır N. Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. JISE. 2023;7:88–105.
MLA Avcı, Adem and Nurettin Acır. “Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models”. Journal of Innovative Science and Engineering, vol. 7, no. 1, 2023, pp. 88-105, doi:10.38088/jise.1206920.
Vancouver Avcı A, Acır N. Remaining Useful Life Estimation via Cascaded Self-Attention and ResNet Models. JISE. 2023;7(1):88-105.
 Download Cover Image


Creative Commons License

The works published in Journal of Innovative Science and Engineering (JISE) are licensed under a  Creative Commons Attribution-NonCommercial 4.0 International License.