Capture
OpenCV reads frames from the local webcam.
WEBCAMLocal computer-vision prototype
A hand gesture.
A desktop action.
A Python desktop experiment that follows hand landmarks from a webcam and maps a few gestures to the operating system pointer.
Not a browser app. Camera, preview window, and pointer control run on your own computer.
THE PIPELINE
From camera frame
to pointer movement.
This is an authored workflow diagram based on the repository implementation, not a captured application screenshot.
Track
MediaPipe estimates hand landmarks in each frame.
HAND LANDMARKSMap
Gesture and coordinate logic maps movement and pinches.
GESTURE LOGICAct
PyAutoGUI sends pointer, click, and scroll actions.
DESKTOP CONTROLLandmarks also appear in the local camera preview
CONTROLS
A small set of deliberate actions.
Move
Place the index fingertip inside the blue tracking box to move the pointer, with smoothing.
Pinch
Bring thumb and index finger together for a click with a short repeat delay.
Scroll
Raise index and middle fingers, then move vertically to scroll.
BEFORE YOU RUN IT
Built for a local desktop.
- Python 3.9+ with project dependencies installed
- A webcam and an interactive graphical desktop
- Camera access; on macOS, Accessibility permission for pointer control
- The hand-landmarker model downloaded by the provided script
LOCAL PROCESSING
Frames stay on your machine.
The project code processes webcam frames locally. Desktop actions are disabled until you press t in the camera window. Press q to exit. This static page has no webcam access, API, or gesture simulator.
TRY THE DESKTOP PROJECT