Research Article
BibTex RIS Cite

A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines

Year 2024, Volume: 8 Issue: 2, 179 - 198

Mustafa Çoban , Gökhan Gelen

https://doi.org/10.38088/jise.1482853

Abstract

Automated guided vehicles are transportation systems that are widely used in factories, warehouses, and distribution centers. It is of great importance to ensure the control and coordination of vehicles for safe and efficient transportation in multi-vehicle systems. In this study, a control strategy is proposed to enforce collision avoidance of automated guided vehicles operating in a shared zone and overlapping route environment. In the proposed method, while finite state machines are used to model the movement of automated guided vehicles in the environment, the Q-learning method, one of the most common reinforcement learning algorithms, is used for collision avoidance. The presented approach uses the decentralized node-based approach to reduce computational complexity. The proposed method has been validated through simulation performed with vehicle applications that can move both unidirectional and bidirectional. The simulation results show that our presented approach can avoid potential collisions and greatly increase overall efficiency.

Keywords

Automated guided vehicle Collision avoidance Finite state machines Reinforcement learning

References

  • [1] Cassandras, C. G., and Lafortune, S. (1999). Discrete event systems: The state of the art and new directions. Applied and Computational Control, Signals, and Circuits: Volume 1: 1-65.
  • [2] Fanti, M. P. (2002). A deadlock avoidance strategy for AGV systems modelled by coloured Petri nets. In Sixth International Workshop on Discrete Event Systems, 61-66.
  • [3] Fanti, M. P., and Zhou, M. (2004). Deadlock control methods in automated manufacturing systems. IEEE Transactions on systems, man, and cybernetics-part A: systems and humans, 34(1): 5-22.
  • [4] Wu, N., and Zhou, M. (2005). Modeling and deadlock avoidance of automated manufacturing systems with multiple automated guided vehicles. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 35(6): 1193-1202.
  • [5] Manca, S., Fagiolini, A., and Pallottino, L. (2011). Decentralized coordination system for multiple agvs in a structured environment. IFAC Proceedings Volumes, 44(1): 6005-6010.
  • [6] Hernández Martínez, E. G., Pérez Sampieri, J. C., and Aranda Bricaire, E. (2013). Supervisory control of AGV’s for flexible manufacturing cells. Congreso Nacional de Control Automatico, Ensenada-Baja California-Meksika, 366-371.
  • [7] Fanti, M. P., Mangini, A. M., Pedroncelli, G., and Ukovich, W. (2015). Decentralized deadlock-free control for AGV systems. In 2015 American Control Conference (ACC), 2414-2419.
  • [8] Fanti, M. P., Mangini, A. M., Pedroncelli, G., and Ukovich, W. (2018). A decentralized control strategy for the coordination of AGV systems. Control Engineering Practice, 70: 86-97.
  • [9] Wan, Y., Luo, J., Zhang, Q., Wu, W., Huang, Y., and Zhou, M. (2018). Controller design for avoiding collisions in automated guided vehicle systems via labeled petri nets. IFAC-PapersOnLine, 51(7): 139-144.
  • [10] Małopolski, W. (2018). A sustainable and conflict-free operation of AGVs in a square topology. Computers & Industrial Engineering, 126: 472-481.
  • [11] Zając, J., and Małopolski, W. (2021). Structural on-line control policy for collision and deadlock resolution in multi-AGV systems. Journal of Manufacturing Systems, 60: 80-92.
  • [12] Luo, J., Wan, Y., Wu, W., and Li, Z. (2019). Optimal Petri-net controller for avoiding collisions in a class of automated guided vehicle systems. IEEE Transactions on Intelligent Transportation Systems, 21(11): 4526-4537.
  • [13] Chen, X., Xing, Z., Feng, L., Zhang, T., Wu, W., and Hu, R. (2022). An ETCEN-based motion coordination strategy avoiding active and passive deadlocks for multi-AGV system. IEEE Transactions on Automation Science and Engineering, 20(2): 1364-1377.
  • [14] Chen, X., Wu, W., and Hu, R. (2022). A Novel Multi-AGV Coordination Strategy Based on the Combination of Nodes and Grids. IEEE Robotics and Automation Letters, 7(3): 6218-6225.
  • [15] Maza, S. (2023). Hybrid supervisory-based architecture for robust control of Bi-directional AGVs. Computers in Industry, 144: 103797.
  • [16] Jeon, S. M., Kim, K. H., and Kopfer, H. (2011). Routing automated guided vehicles in container terminals through the Q-learning technique. Logistics Research, 3: 19-27.
  • [17] Nagayoshi, M., Elderton, S. J., Sakakibara, K., and Tamaki, H. (2017). Reinforcement Learning Approach for Adaptive Negotiation-Rules Acquisition in AGV Transportation Systems. Journal of Advanced Computational Intelligence and Intelligent Informatics, 21(5): 948-957.
  • [18] Xue, T., Zeng, P., and Yu, H. (2018). A reinforcement learning method for multi-AGV scheduling in manufacturing. In 2018 IEEE international conference on industrial technology (ICIT), 1557-1561.
  • [19] Hu, H., Jia, X., He, Q., Fu, S., and Liu, K. (2020). Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0. Computers & Industrial Engineering, 149: 106749.
  • [20] Jestel, C., Surmann, H., Stenzel, J., Urbann, O., and Brehler, M. (2021). Obtaining robust control and navigation policies for multi-robot navigation via deep reinforcement learning. In 2021 7th International Conference on Automation, Robotics and Applications (ICARA), 48-54.
  • [21] Zhou, P., Lin, L., and Kim, K. H. (2023). Anisotropic Q-learning and waiting estimation based real-time routing for automated guided vehicles at container terminals. Journal of Heuristics: 1-22.
  • [22] Zhang, H., Luo, J., Lin, X., Tan, K., and Pan, C. (2021). Dispatching and path planning of automated guided vehicles based on petri nets and deep reinforcement learning. In 2021 IEEE International Conference on Networking, Sensing and Control (ICNSC) Vol. 1, 1-6.
  • [23] Choi, H. B., Kim, J. B., Ji, C. H., Ihsan, U., Han, Y. H., Oh, S. W., Kim, K. H. and Pyo, C. S. (2022). Marl-based optimal route control in multi-agv warehouses. In 2022 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), 333-338.
  • [24] Sagar, K. V., and Jerald, J. (2022). Real-time automated guided vehicles scheduling with Markov decision process and double Q-learning algorithm. Materials Today: Proceedings, 64: 279-284.
  • [25] Zhang, Z., Chen, J., and Guo, Q. (2023). Application of Automated Guided Vehicles in Smart Automated Warehouse Systems: A Survey. CMES-Computer Modeling in Engineering & Sciences, 134(3).
  • [26] Hu, H., Yang, X., Xiao, S., and Wang, F. (2023). Anti-conflict AGV path planning in automated container terminals based on multi-agent reinforcement learning. International Journal of Production Research, 61(1): 65-80.

Year 2024, Volume: 8 Issue: 2, 179 - 198

Mustafa Çoban , Gökhan Gelen

https://doi.org/10.38088/jise.1482853

Abstract

References

  • [1] Cassandras, C. G., and Lafortune, S. (1999). Discrete event systems: The state of the art and new directions. Applied and Computational Control, Signals, and Circuits: Volume 1: 1-65.
  • [2] Fanti, M. P. (2002). A deadlock avoidance strategy for AGV systems modelled by coloured Petri nets. In Sixth International Workshop on Discrete Event Systems, 61-66.
  • [3] Fanti, M. P., and Zhou, M. (2004). Deadlock control methods in automated manufacturing systems. IEEE Transactions on systems, man, and cybernetics-part A: systems and humans, 34(1): 5-22.
  • [4] Wu, N., and Zhou, M. (2005). Modeling and deadlock avoidance of automated manufacturing systems with multiple automated guided vehicles. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 35(6): 1193-1202.
  • [5] Manca, S., Fagiolini, A., and Pallottino, L. (2011). Decentralized coordination system for multiple agvs in a structured environment. IFAC Proceedings Volumes, 44(1): 6005-6010.
  • [6] Hernández Martínez, E. G., Pérez Sampieri, J. C., and Aranda Bricaire, E. (2013). Supervisory control of AGV’s for flexible manufacturing cells. Congreso Nacional de Control Automatico, Ensenada-Baja California-Meksika, 366-371.
  • [7] Fanti, M. P., Mangini, A. M., Pedroncelli, G., and Ukovich, W. (2015). Decentralized deadlock-free control for AGV systems. In 2015 American Control Conference (ACC), 2414-2419.
  • [8] Fanti, M. P., Mangini, A. M., Pedroncelli, G., and Ukovich, W. (2018). A decentralized control strategy for the coordination of AGV systems. Control Engineering Practice, 70: 86-97.
  • [9] Wan, Y., Luo, J., Zhang, Q., Wu, W., Huang, Y., and Zhou, M. (2018). Controller design for avoiding collisions in automated guided vehicle systems via labeled petri nets. IFAC-PapersOnLine, 51(7): 139-144.
  • [10] Małopolski, W. (2018). A sustainable and conflict-free operation of AGVs in a square topology. Computers & Industrial Engineering, 126: 472-481.
  • [11] Zając, J., and Małopolski, W. (2021). Structural on-line control policy for collision and deadlock resolution in multi-AGV systems. Journal of Manufacturing Systems, 60: 80-92.
  • [12] Luo, J., Wan, Y., Wu, W., and Li, Z. (2019). Optimal Petri-net controller for avoiding collisions in a class of automated guided vehicle systems. IEEE Transactions on Intelligent Transportation Systems, 21(11): 4526-4537.
  • [13] Chen, X., Xing, Z., Feng, L., Zhang, T., Wu, W., and Hu, R. (2022). An ETCEN-based motion coordination strategy avoiding active and passive deadlocks for multi-AGV system. IEEE Transactions on Automation Science and Engineering, 20(2): 1364-1377.
  • [14] Chen, X., Wu, W., and Hu, R. (2022). A Novel Multi-AGV Coordination Strategy Based on the Combination of Nodes and Grids. IEEE Robotics and Automation Letters, 7(3): 6218-6225.
  • [15] Maza, S. (2023). Hybrid supervisory-based architecture for robust control of Bi-directional AGVs. Computers in Industry, 144: 103797.
  • [16] Jeon, S. M., Kim, K. H., and Kopfer, H. (2011). Routing automated guided vehicles in container terminals through the Q-learning technique. Logistics Research, 3: 19-27.
  • [17] Nagayoshi, M., Elderton, S. J., Sakakibara, K., and Tamaki, H. (2017). Reinforcement Learning Approach for Adaptive Negotiation-Rules Acquisition in AGV Transportation Systems. Journal of Advanced Computational Intelligence and Intelligent Informatics, 21(5): 948-957.
  • [18] Xue, T., Zeng, P., and Yu, H. (2018). A reinforcement learning method for multi-AGV scheduling in manufacturing. In 2018 IEEE international conference on industrial technology (ICIT), 1557-1561.
  • [19] Hu, H., Jia, X., He, Q., Fu, S., and Liu, K. (2020). Deep reinforcement learning based AGVs real-time scheduling with mixed rule for flexible shop floor in industry 4.0. Computers & Industrial Engineering, 149: 106749.
  • [20] Jestel, C., Surmann, H., Stenzel, J., Urbann, O., and Brehler, M. (2021). Obtaining robust control and navigation policies for multi-robot navigation via deep reinforcement learning. In 2021 7th International Conference on Automation, Robotics and Applications (ICARA), 48-54.
  • [21] Zhou, P., Lin, L., and Kim, K. H. (2023). Anisotropic Q-learning and waiting estimation based real-time routing for automated guided vehicles at container terminals. Journal of Heuristics: 1-22.
  • [22] Zhang, H., Luo, J., Lin, X., Tan, K., and Pan, C. (2021). Dispatching and path planning of automated guided vehicles based on petri nets and deep reinforcement learning. In 2021 IEEE International Conference on Networking, Sensing and Control (ICNSC) Vol. 1, 1-6.
  • [23] Choi, H. B., Kim, J. B., Ji, C. H., Ihsan, U., Han, Y. H., Oh, S. W., Kim, K. H. and Pyo, C. S. (2022). Marl-based optimal route control in multi-agv warehouses. In 2022 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), 333-338.
  • [24] Sagar, K. V., and Jerald, J. (2022). Real-time automated guided vehicles scheduling with Markov decision process and double Q-learning algorithm. Materials Today: Proceedings, 64: 279-284.
  • [25] Zhang, Z., Chen, J., and Guo, Q. (2023). Application of Automated Guided Vehicles in Smart Automated Warehouse Systems: A Survey. CMES-Computer Modeling in Engineering & Sciences, 134(3).
  • [26] Hu, H., Yang, X., Xiao, S., and Wang, F. (2023). Anti-conflict AGV path planning in automated container terminals based on multi-agent reinforcement learning. International Journal of Production Research, 61(1): 65-80.
There are 26 citations in total.

Details

Primary Language English
Subjects Control Engineering, Mechatronics and Robotics (Other)
Journal Section Research Articles
Authors

Mustafa Çoban 0000-0002-6508-5901

Gökhan Gelen 0000-0002-2780-3386

Early Pub Date December 11, 2024
Publication Date
Submission Date May 13, 2024
Acceptance Date July 4, 2024
Published in Issue Year 2024Volume: 8 Issue: 2

Cite

APA Çoban, M., & Gelen, G. (2024). A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines. Journal of Innovative Science and Engineering, 8(2), 179-198. https://doi.org/10.38088/jise.1482853
AMA Çoban M, Gelen G. A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines. JISE. December 2024;8(2):179-198. doi:10.38088/jise.1482853
Chicago Çoban, Mustafa, and Gökhan Gelen. “A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines”. Journal of Innovative Science and Engineering 8, no. 2 (December 2024): 179-98. https://doi.org/10.38088/jise.1482853.
EndNote Çoban M, Gelen G (December 1, 2024) A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines. Journal of Innovative Science and Engineering 8 2 179–198.
IEEE M. Çoban and G. Gelen, “A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines”, JISE, vol. 8, no. 2, pp. 179–198, 2024, doi: 10.38088/jise.1482853.
ISNAD Çoban, Mustafa - Gelen, Gökhan. “A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines”. Journal of Innovative Science and Engineering 8/2 (December 2024), 179-198. https://doi.org/10.38088/jise.1482853.
JAMA Çoban M, Gelen G. A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines. JISE. 2024;8:179–198.
MLA Çoban, Mustafa and Gökhan Gelen. “A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines”. Journal of Innovative Science and Engineering, vol. 8, no. 2, 2024, pp. 179-98, doi:10.38088/jise.1482853.
Vancouver Çoban M, Gelen G. A New Collision Avoidance Approach for Automated Guided Vehicle Systems Based on Finite State Machines. JISE. 2024;8(2):179-98.
 Download Cover Image


Creative Commons License

The works published in Journal of Innovative Science and Engineering (JISE) are licensed under a  Creative Commons Attribution-NonCommercial 4.0 International License.