Gesture Painter

Paint on a canvas using only your hand. No mouse, no stylus, no touching anything! Your index fingertip becomes a brush, tracked in real time by the gesture detector.

Open In Jupyter K-12

Gesture Controls

Gesture Action
Pointing Up (index finger extended) Draw — your fingertip leaves a trail
Open Palm Lift pen — move without drawing
Closed Fist Clear — wipes the canvas clean

How It Works

Each frame, the app reads the position of your index fingertip (cv.HAND.INDEX_FINGER_TIP). When the gesture is Pointing_Up, it draws a line from the previous fingertip position to the current one.

All line segments are stored in a list and redrawn every frame on top of the live camera feed, so your painting persists as the video updates underneath.

Choose your brush color and size below, then run the cell.

Paint!

Set your brush color and size, then run the cell. To change the brush mid-session: click Stop (■), adjust the controls, and run again.

Click Stop when you’re done.

import cv
import graphics
import time

BRUSH_COLOR = "#ff4500" #@param ["#ff4500", "#00bfff", "#00ff88", "#ffff00", "#ffffff", "#ff69b4"]
BRUSH_SIZE  = 6         #@param {type:"slider", min:2, max:20, step:1}

canvas   = graphics.canvas()
camera   = cv.start_camera(canvas)
detector = cv.start_gesture_detector(camera)
ctx      = canvas.get_context('2d')

strokes        = []
last_pos       = None
clear_cooldown = 0

try:
  while True:
    detections = detector.get_detections()
    canvas.draw_hands(detections)

    if detections:
      hand    = detections[0]
      tip     = hand['landmarks'][cv.HAND.INDEX_FINGER_TIP]
      gesture = hand['gesture']

      if clear_cooldown > 0:
        clear_cooldown -= 1
        last_pos = None
      elif gesture == 'Pointing_Up':
        if last_pos:
          strokes.append((last_pos[0], last_pos[1],
                  tip['x'], tip['y'], BRUSH_COLOR, BRUSH_SIZE))
        last_pos = (tip['x'], tip['y'])
      elif gesture == 'Closed_Fist':
        strokes.clear()
        ctx.clear_rect(0, 0, 9999, 9999)
        last_pos       = None
        clear_cooldown = 30
      else:
        last_pos = None
    else:
      last_pos = None

    for x1, y1, x2, y2, col, sz in strokes:
      ctx.stroke_style = col
      ctx.line_width   = sz
      ctx.begin_path()
      ctx.move_to(x1, y1)
      ctx.line_to(x2, y2)
      ctx.stroke()

    time.sleep(0.033)
finally:
  detector.stop()
  camera.stop()
  print("Camera stopped.")

What did you create? What’s one thing you’d change or add to make this a more powerful drawing tool?

Ideas to Extend This App

  • Color switching — assign different colors to different gestures (e.g. Victory → blue, Thumb_Up → yellow)
  • Two-hand mode — use num_hands=2 so each hand paints independently
  • Stamps — detect Thumb_Up and paste an image at the fingertip position using graphics.draw_image()
  • Eraser — Open_Palm in eraser mode could draw a filled white circle to erase strokes
  • Undo — keep track of how many strokes were added per gesture and remove the last group on a specific gesture