AI Hand Gesture Mouse Controller

Control your Windows cursor using only your hand and a webcam — no physical mouse required.

Built with Python, MediaPipe, and OpenCV.

---

Demo

Gesture	Action
☝️ Index finger raised	Move cursor
🤌 Thumb + Index pinch	Left click
🤏 Thumb + Middle pinch	Right click
✌️ Index + Middle raised → move up	Scroll up
✌️ Index + Middle raised → move down	Scroll down

---

Requirements

• Windows 10 or Windows 11
• Python 3.12+
• Built-in or USB webcam

---

Installation

# 1. Clone the repository
https://github.com/Alshahriar-07/Hand-tracking-Mouse-Control.git
cd Hand racking Mouse Control

# 2. (Recommended) Create and activate a virtual environment
python -m venv .venv
.venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

---

Usage

pip install -r requirements.txt
python main.py

A preview window will open showing your webcam feed with hand landmarks overlaid.

• Press Q or Esc to quit.

---

How It Works

Webcam frame
    │
    ▼
camera.py          — captures 640×480 frames at 30 FPS (low-latency buffer)
    │
    ▼
hand_tracker.py    — MediaPipe detects 21 landmarks on your right hand
    │
    ▼
gesture_detector.py — classifies landmarks into MOVE / CLICK / SCROLL
    │
    ▼
mouse_controller.py — maps hand position → screen coords → Win32 SetCursorPos
                       + fires click/scroll events via pynput

Cursor movement uses an Exponential Moving Average (EMA) smoother to eliminate jitter without introducing noticeable lag.

---

Project Structure

├── main.py              # Entry point — main loop
├── camera.py            # Webcam capture wrapper
├── hand_tracker.py      # MediaPipe Hands integration
├── gesture_detector.py  # Gesture classification engine
├── mouse_controller.py  # Cursor movement and click dispatch
├── smoothing.py         # EMA cursor smoother
├── config.py            # All tunable constants
├── utils.py             # Shared helpers (FPS counter, clamp, remap)
├── assets/              # Icons and images (future use)
└── requirements.txt     # Pinned Python dependencies

---

Configuration

All tunable parameters live in config.py. No code changes needed for basic tuning.

Parameter	Default	Description
CAMERA_INDEX	0	Webcam index (1, 2… for external cameras)
EMA_ALPHA	0.25	Cursor smoothing — lower = smoother, higher = more responsive
ACTIVE_ZONE_MARGIN	0.15	Dead-zone margin on each frame edge (15 % per side)
PINCH_LEFT_THRESHOLD	0.06	Normalised pinch distance for left click
PINCH_RIGHT_THRESHOLD	0.06	Normalised pinch distance for right click
CLICK_HOLD_FRAMES	2	Frames the pinch must hold before a click fires
CLICK_COOLDOWN_FRAMES	20	Lockout frames after a click to prevent repeats
SCROLL_SPEED	3	Lines scrolled per gesture tick
SHOW_FPS	True	Toggle FPS overlay in the preview window
FLIP_FRAME	True	Mirror the preview (recommended — feels natural)

---

Gesture Detail

Cursor Movement

Raise only your index finger. The tip (landmark 8) is mapped from the camera's active zone to the full screen. Move your hand to move the cursor.

Left Click

Bring your thumb tip and index tip together (pinch). Hold for 2 frames. A cooldown prevents accidental double-clicks.

Right Click

Bring your thumb tip and middle tip together (pinch). Hold for 2 frames. Detected before left-click in the priority chain.

Scroll

Raise both your index and middle fingers. Moving them upward scrolls up; moving them downward scrolls down. Speed is proportional to movement delta.

---

Performance Targets

Metric	Target
Resolution	640 × 480
Frame rate	30–60 FPS
CPU usage	10–20 %
Memory	< 200 MB
Latency	< 50 ms

---

Tech Stack

Library	Purpose
MediaPipe	Real-time 21-point hand landmark detection
OpenCV	Webcam capture and preview rendering
pynput	Cross-process mouse click and scroll events
NumPy	Numerical operations
PyAutoGUI	Screen resolution detection

---

Troubleshooting

Camera does not open Change CAMERA_INDEX in config.py to 1 or 2 if you have multiple cameras.

Cursor is too shaky Lower EMA_ALPHA in config.py (e.g. 0.15).

Cursor feels laggy Raise EMA_ALPHA in config.py (e.g. 0.40).

Clicks trigger too easily Lower PINCH_LEFT_THRESHOLD or PINCH_RIGHT_THRESHOLD (e.g. 0.04).

Wrong hand is tracked Ensure you are showing your right hand to the camera. The tracker ignores the left hand.

---

Roadmap

• Drag & Drop
• Double Click gesture
• Cursor Pause gesture
• Volume & Brightness control
• Screenshot gesture
• Multi-monitor support
• Calibration wizard
• Custom gesture mapping
• PySide6 settings window
• ONNX Runtime AI optimisation

---

License

MIT License — see LICENSE for details.