AI Virtual Mouse Using Hand Gestures

International Journal of Emerging Research in Science, Engineering, and Management
Vol. 2, Issue 1, pp. 288-297, January 2026.

https://doi.org/10.58482/ijersem.v2i1.39

This work is licensed under a Creative Commons Attribution 4.0 International License.

AI Virtual Mouse Using Hand Gestures

Full Text PDF

B.Hima Bindu

Godugu Thejaswini

Kovuru Ramalakshmi

Konga Uma Maheswari

Sompalli Pavithra

Nemalla Venkateswara

Department of CSE, Siddartha Institute of Science and Technology, Puttur, India.

Abstract: This represents an increasing trend towards natural and touchless interfaces, driven by leaps in computer vision and AI. The mouse device itself imposes serious limitations, above all in sterile environments, accessibility-focused scenarios, and situations that absolutely require hands-free control. Deep learning-based gesture recognition overcomes this through virtual mouse systems-interpreting hand movements captured via a webcam into cursor actions. With an AI-enabled Virtual Mouse System, intuitive, touchless movements, clicks, drag, and scroll operations are provided through simple hand gestures. It enhances the sense of accessibility, hygiene, and allows for a seamless experience that proves quite valuable in healthcare, gaming, AR/VR applications, and other smart and interactive settings.

Keywords: Hand Gesture Recognition, AI Virtual Mouse, Computer Vision, Touchless Human–Computer Interaction, MediaPipe Hands.

References: 

  1. A. Haria, A. Subramanian, N. Asokkumar, S. Poddar, and J. S. Nayak, “Hand gesture recognition for human computer interaction,” Procedia Computer Science, vol. 115, pp. 367–374, Jan. 2017, doi: 10.1016/j.procs.2017.09.092.
  2. P. U. Vignesh, M. Thaqib-Ul-Rahman, S. Dheeraj, T. S. C. Reddy, and M. A. Uddin, “AI virtual mouse using hand gesture recognition,” International Research Journal on Advanced Engineering Hub (IRJAEH), vol. 3, no. 05, pp. 2259–2263, May 2025, doi: 10.47392/irjaeh.2025.0332.
  3. A. Tang, K. Lu, Y. Wang, J. Huang, and H. Li, “A Real-Time hand posture recognition system using deep neural networks,” ACM Transactions on Intelligent Systems and Technology, vol. 6, no. 2, pp. 1–23, Mar. 2015, doi: 10.1145/2735952.
  4. X. Lou, “Vision-based hand gesture recognition technology,” Applied and Computational Engineering, vol. 141, no. 1, pp. 54–59, Mar. 2025, doi: 10.54254/2755-2721/2025.21696.
  5. D. D. Doyle, A. L. Jennings, and J. T. Black, “Optical flow background estimation for real-time pan/tilt camera object tracking,” Measurement, vol. 48, pp. 195–207, Oct. 2013, doi: 10.1016/j.measurement.2013.10.025.
  6. M. Wu, “Gesture Recognition Based on Deep Learning: A review,” ICST Transactions on e-Education and e-Learning, vol. 10, Mar. 2024, doi: 10.4108/eetel.5191.
  7. P. U. Vignesh, M. Thaqib-Ul-Rahman, S. Dheeraj, T. S. C. Reddy, and M. A. Uddin, “AI virtual mouse using hand gesture recognition,” International Research Journal on Advanced Engineering Hub (IRJAEH), vol. 3, no. 05, pp. 2259–2263, May 2025, doi: 10.47392/irjaeh.2025.0332.
  8. J. Ao, S. Liang, T. Yan, R. Hou, Z. Zheng, and J. Ryu, “Overcoming the effect of muscle fatigue on gesture recognition based on sEMG via generative adversarial networks,” Expert Systems With Applications, vol. 238, p. 122304, Nov. 2023, doi: 10.1016/j.eswa.2023.122304.
  9. R. Matlani, R. Dadlani, S. Dumbre, S. Mishra, and A. Tewari, “Virtual Mouse using Hand Gestures,” 2021 International Conference on Technological Advancements and Innovations (ICTAI), pp. 340–345, Nov. 2021, doi: 10.1109/ictai53825.2021.9673251.
  10. R. P. Sharma and G. K. Verma, “Human Computer Interaction using Hand Gesture,” Procedia Computer Science, vol. 54, pp. 721–727, Jan. 2015, doi: 10.1016/j.procs.2015.06.085.
  11. K. Żywanowski et al., “Vision-based hand pose estimation methods for Augmented Reality in industry: Crowdsourced evaluation on HoloLens 2,” Computers in Industry, vol. 171, p. 104328, Jun. 2025, doi: 10.1016/j.compind.2025.104328.
  12. S. Panagou, F. Fruggiero, and A. Lambiase, “Human gesture system in Human Robot Interaction for reliability analysis,” Procedia Computer Science, vol. 200, pp. 1788–1795, Jan. 2022, doi: 10.1016/j.procs.2022.01.379.
  13. F. Zhang et al., “MediaPipe Hands: On-device real-time hand tracking,” arXiv (Cornell University), Jun. 2020, doi: 10.48550/arxiv.2006.10214.
Download PDF

This article is part of Volume 2, Issue 1 (January 2026)