Object Detection with Computer Vision

In this notebook, you’ll use a neural network to detect and label everyday objects in real time. Unlike face detection, which only looks for one thing, an object detector can recognize 80 different categories — all at the same time.

From Face Detection to Object Detection

In the face detection notebook, the model was trained to find one thing: faces. An object detector is more general — it’s trained to find dozens of different object categories at once.

This platform uses EfficientDet-Lite0, a compact but powerful model trained on the COCO dataset. COCO contains 80 everyday object categories, including:

People, chairs, laptops, phones, cups, books
Cars, bikes, dogs, cats, birds
And many more…

Each detection still returns a bounding box and a confidence score — but now also includes a label telling you what was found.

Step 1: Live Object Detection

The cell below starts your camera and runs the object detector in real time. Bounding boxes will appear around anything the model recognizes, labeled with the object type.

Point the camera around the room to see what it finds. Click Allow when your browser asks for webcam access, then click Stop (■) when you’re ready to move on.

import cv
import graphics

canvas = graphics.canvas()
camera = cv.start_camera(canvas)
detector = cv.start_object_detector(camera)

try:
  while True:
    detections = detector.get_detections()
    canvas.draw_bounding_boxes(detections)
finally:
  detector.stop()
  camera.stop()
  print("Camera stopped.")

Step 2: Explore Detection Data

The cell below runs the same live loop but also prints the raw detection data once per second. Each detection has a type (the object label), position, size, and confidence score.

Try pointing the camera at different objects around the room and watch the output update. Click Stop (■) when you’re done.

import cv
import graphics

canvas = graphics.canvas()
camera = cv.start_camera(canvas)
detector = cv.start_object_detector(camera)

frame = 0
try:
  while True:
    detections = detector.get_detections()
    canvas.draw_bounding_boxes(detections)
    if frame % 240 == 0:
      print(f"--- {len(detections)} object(s) detected ---")
      for i, obj in enumerate(detections):
        print(f"  {obj['type']}: ({obj['x']}, {obj['y']})  "
            f"size={obj['w']}x{obj['h']}  confidence={obj['confidence']:.1%}")
    frame += 1
finally:
  detector.stop()
  camera.stop()
  print("Camera stopped.")

Step 3: Experiment — Confidence Threshold

A confidence threshold controls how certain the model must be before a detection is shown.

A low threshold finds more objects, but some may be wrong
A high threshold only shows objects the model is very sure about

Set the slider to your desired threshold, then run the cell. The loop filters detections in real time using that value. Click Stop (■) when you’re done.

import cv
import graphics

CONFIDENCE_THRESHOLD = 0.5 #@param {type:"slider", min:0.1, max:1.0, step:0.05}

canvas = graphics.canvas()
camera = cv.start_camera(canvas)
detector = cv.start_object_detector(camera)

try:
  while True:
    detections = detector.get_detections()
    filtered = [d for d in detections if d['confidence'] >= CONFIDENCE_THRESHOLD]
    canvas.draw_bounding_boxes(filtered)
finally:
  detector.stop()
  camera.stop()
  print("Camera stopped.")

What objects did the detector find? Were there any surprises — things it labeled incorrectly or missed entirely?

Check for Understanding

{ “question_type”: “true_false”, “question”: “An object detector can find more than one object in a single frame.”, “answer”: “True”, “submitted_answer”: “” }

{ “question_type”: “multiple_choice”, “question”: “How many object categories can EfficientDet-Lite0 detect?”, “options”: [ { “key”: “a”, “text”: “10” }, { “key”: “b”, “text”: “40” }, { “key”: “c”, “text”: “80” }, { “key”: “d”, “text”: “200” } ], “answer”: “c”, “submitted_answer”: “” }

{ “question_type”: “true_false”, “question”: “The type field in an object detection tells you how many objects were found.”, “answer”: “False”, “submitted_answer”: “” }

{ “question_type”: “multiple_choice”, “question”: “What is drawn around each object that the detector finds?”, “options”: [ { “key”: “a”, “text”: “A skeleton” }, { “key”: “b”, “text”: “A colored pixel mask” }, { “key”: “c”, “text”: “A bounding box” }, { “key”: “d”, “text”: “A dot at the center” } ], “answer”: “c”, “submitted_answer”: “” }