Controlling a 4 Degree of Freedom (4 DoF) Robot Arm with Hand Gestures Using Computer Vision Technology for Manufacturing Processes

Royyan Naufal Fauzan, Resa Pramudita

Abstract

The increasing complexity of tasks in industrial environments requiring higher efficiency and precision is a major issue for industrial workers. To address these challenges, technology is needed to speed up the production process and improve quality. As part of artificial intelligence systems in industrial environments, robot arms are a solution that can be considered. The development of a robot arm that can be controlled with hand gestures through computer vision technology becomes a feasible solution. 4 Degree of Freedom (DoF) robot arm is chosen as the experimental platform for this research. The research method used in this study is research and development. Utilizing computer vision algorithms to track and detect user hand movements, the user can interact with the robot arm without using additional control devices. Hand object segmentation, feature extraction, and mapping to robot coordinates are parts of the hand gesture recognition process. The results of this research are expected to help develop a responsive and user-friendly robot arm control system. The system was tested under various usage conditions and achieved an average accuracy rate of 93%. With these research findings, it is expected to help industries develop new optimization solutions.

Full Text:

PDF

References

E. Demir, K. Yildiz, O. Demir, and E. E. Ulku, “Computer Vision Based Intelligent 3D CNC Machines,” Proc. - 2020 Innov. Intell. Syst. Appl. Conf. ASYU 2020, 2020, doi: 10.1109/ASYU50717.2020.9259826.

H. Song, “The Application of Computer Vision in Responding to the Emergencies of Autonomous Driving,” Proc. - 2020 Int. Conf. Comput. Vision, Image Deep Learn. CVIDL 2020, no. Cvidl, pp. 1–5, 2020, doi: 10.1109/CVIDL51233.2020.00008.

Z. Wang, D. Chen, and P. Xiao, “Design of a voice control 6DOF grasping robotic arm based on ultrasonic sensor, computer vision and alexa voice assistance,” Proc. - 10th Int. Conf. Inf. Technol. Med. Educ. ITME 2019, pp. 649–654, 2019, doi: 10.1109/ITME.2019.00150.

Y. Li and Y. Zhang, “Application research of computer vision technology in automation,” Proc. - 2020 Int. Conf. Comput. Inf. Big Data Appl. CIBDA 2020, pp. 374–377, 2020, doi: 10.1109/CIBDA50819.2020.00090.

W. B. Dou, W. H. Chin, and N. Kubota, “Hand gesture communication using deep learning based on relevance theory,” 2020 Jt. 11th Int. Conf. Soft Comput. Intell. Syst. 21st Int. Symp. Adv. Intell. Syst. SCIS-ISIS 2020, pp. 6–10, 2020, doi: 10.1109/SCISISIS50064.2020.9322784.

J. L. Raheja, R. Shyam, U. Kumar, and P. B. Prasad, “Real-time robotic hand control using hand gestures,” ICMLC 2010 - 2nd Int. Conf. Mach. Learn. Comput., pp. 12–16, 2010, doi: 10.1109/ICMLC.2010.12.

H. Rahmani, S. M. Taheri, and E. Yargholi, “From Brain Decoding to Brain-Driven Computer Vision,” Iran. Conf. Mach. Vis. Image Process. MVIP, vol. 2020-Febru, 2020, doi: 10.1109/MVIP49855.2020.9116906.

A. G. Spilling, A. R. Nix, M. A. Beach, and T. J. Harrold, “Electronics and Communication Engineering Journal,” pp. 1–3, 1999, [Online]. Available: citeseer.ist.psu.edu/hong99group.html.

P. Sihombing, M. B. Rifky, H. Herriyance, and E. Elviwani, “Robotic Arm Controlling Based on Fingers and Hand Gesture,” Mecn. 2020 - Int. Conf. Mech. Electron. Comput. Ind. Technol., pp. 40–45, 2020, doi: 10.1109/MECnIT48290.2020.9166592.

C. Orhei, M. Mocofan, S. Vert, and R. Vasiu, “End-to-End Computer Vision Framework,” 2020 14th Int. Symp. Electron. Telecommun. ISETC 2020 - Conf. Proc., 2020, doi: 10.1109/ISETC50328.2020.9301078.

J. Dastur and A. Khawaja, “Robotic arm actuation with 7 DOF using Haar classifier gesture recognition,” 2010 2nd Int. Conf. Comput. Eng. Appl. ICCEA 2010, vol. 1, pp. 27–29, 2010, doi: 10.1109/ICCEA.2010.13.

M. Wongphati, Y. Kanai, H. Osawa, and M. Imai, “Give me a hand - How users ask a robotic arm for help with gestures,” Proc. - 2012 IEEE Int. Conf. Cyber Technol. Autom. Control. Intell. Syst. CYBER 2012, pp. 64–68, 2012, doi: 10.1109/CYBER.2012.6392528.

H. He and Y. Dan, “The research and design of Smart mobile robotic arm based on gesture controlled,” Int. Conf. Adv. Mechatron. Syst. ICAMechS, vol. 2020-Decem, pp. 308–312, 2020, doi: 10.1109/ICAMechS49982.2020.9310156.

J. Sepúlveda and S. A. Velastín, “F1 score assesment of Gaussian mixture background subtraction algorithms using the MuHAVi dataset,” IET Semin. Dig., vol. 2015, no. 5, pp. 1–6, 2015, doi: 10.1049/ic.2015.0106.

R. Mardiyanto et al., “Robotic Arm Controlling Based on Fingers and Hand Gesture,” Proc. - 10th Int. Conf. Inf. Technol. Med. Educ. ITME 2019, vol. 2020-Decem, no. 5, pp. 12–16, 2020, doi: 10.1109/SCISISIS50064.2020.9322784.

X. Zhang and S. Xu, “Research on Image Processing Technology of Computer Vision Algorithm,” Proc. - 2020 Int. Conf. Comput. Vision, Image Deep Learn. CVIDL 2020, no. Cvidl, pp. 122–124, 2020, doi: 10.1109/CVIDL51233.2020.00030.

P. Karuppiah, H. Metalia, and K. George, “Automation of a wheelchair mounted robotic arm using computer vision interface,” I2MTC 2018 - 2018 IEEE Int. Instrum. Meas. Technol. Conf. Discov. New Horizons Instrum. Meas. Proc., pp. 1–5, 2018, doi: 10.1109/I2MTC.2018.8409518.

I. B. Çelik and M. Kuntalp, “Development of a robotic-arm controller by using hand gesture recognition,” INISTA 2012 - Int. Symp. Innov. Intell. Syst. Appl., 2012, doi: 10.1109/INISTA.2012.6246985.

S. Gazzah and O. Bencharef, “A Survey on how computer vision can response to urgent need to contribute in COVID-19 pandemics,” 2020 Int. Conf. Intell. Syst. Comput. Vision, ISCV 2020, 2020, doi: 10.1109/ISCV49265.2020.9204043.

R. Mardiyanto, M. F. R. Utomo, D. Purwanto, and H. Suryoatmojo, “Development of hand gesture recognition sensor based on accelerometer and gyroscope for controlling arm of underwater remotely operated robot,” 2017 Int. Semin. Intell. Technol. Its Appl. Strength. Link Between Univ. Res. Ind. to Support ASEAN Energy Sect. ISITIA 2017 - Proceeding, vol. 2017-Janua, pp. 329–333, 2017, doi: 10.1109/ISITIA.2017.8124104.

DOI: https://doi.org/10.20961/jeeict.6.2.90823

Refbacks

  • There are currently no refbacks.