OpenCV on the Pi: Your First Computer Vision Stack

The trap is thinking you can treat OpenCV on a Pi the same way you treat it on your MacBook. You open a tutorial, run pip install opencv-python, wait ten minutes for it to compile, and then call cv2.imshow() on a headless Pi sitting in a closet. The script crashes because there's no display server. You spend an hour installing X11 dependencies you don't need, get it working, and now your Pi is burning 200 MB of RAM on GUI libraries that will never render a single pixel in production.

I install opencv-python-headless on every Pi that doesn't have a monitor attached. That's most of them. The headless build strips out all the GUI and video codec dependencies, dropping the install size and the runtime memory footprint significantly. The only time I reach for the full opencv-python package is when I'm actively developing on a Pi with a desktop environment and I want imshow() windows for debugging.

The distinction matters more than it seems. On a Pi 4 with 2 GB of RAM, those GUI libraries can eat 15–20% of your available memory before you've processed a single frame. On a Pi Zero 2 W with 512 MB, that overhead is the difference between a project that runs and one that swaps itself to death.

Installing OpenCV the Right Way

Two packages, two use cases. Pick one:

# For headless Pi (production, no monitor)
pip install opencv-python-headless

# For Pi with desktop (development, debugging with windows)
pip install opencv-python

Verify it works:

import cv2
print(cv2.__version__)
# Should print something like 4.9.0 or 4.10.0

If that import succeeds, you have a working OpenCV installation. No cmake. No building from source. No hours-long compilation. The pre-built wheels handle the ARM cross-compilation for you — this wasn't always the case, and if you find a tutorial telling you to build OpenCV from source on a Pi, that tutorial is from 2019 and you should close the tab.

The Image as a NumPy Array

Every image in OpenCV is a NumPy array. That single fact unlocks more capability than any API documentation. When you load an image, you get a three-dimensional array: height, width, and channels. The shape tells you everything:

import cv2
import numpy as np

img = cv2.imread('/home/pi/test_image.jpg')
print(img.shape)    # (480, 640, 3) → height=480, width=640, channels=3
print(img.dtype)    # uint8 → values 0–255

The axis order — height first, then width — trips up every developer on their first day. It's (rows, columns, channels), which is (y, x, channels). If you're used to thinking (width, height) from web development or UI frameworks, you need to flip that mental model now.

Accessing a single pixel returns its BGR values:

pixel = img[100, 200]  # row 100, column 200
print(pixel)           # [142, 87, 53] → [Blue, Green, Red]

Yes — Blue, Green, Red. Not RGB.

BGR, Not RGB — OpenCV's Historical Quirk

This is the single most common source of color bugs in OpenCV code, and you will hit it. Not "might." Will. OpenCV uses BGR channel ordering by default. When you load an image with imread(), the channels are Blue, Green, Red — the reverse of what every other library expects. This isn't a bug. It's a design decision from 1999 when Intel created the library, and BGR was the native format of certain capture cards and Windows bitmap headers.

The reason this matters on the Pi specifically is that you're almost certainly going to pass images between multiple libraries. You might capture with picamera2 (which outputs RGB), process with OpenCV (which expects BGR), and then save or stream the result to a web dashboard (which expects RGB again). Every boundary crossing is a potential color swap. On a desktop you'd notice immediately because the preview window shows wrong colors. On a headless Pi, the first human who sees the output is the one reviewing saved images hours later — by which time you've processed hundreds of frames with inverted colors.

The conversion is one line:

# OpenCV BGR → standard RGB (for matplotlib, PIL, web)
rgb_image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

# If you need to go back
bgr_image = cv2.cvtColor(rgb_image, cv2.COLOR_RGB2BGR)

I've seen this pattern where a team builds an entire face detection pipeline, gets it working perfectly in OpenCV, then feeds the results to a web dashboard that displays the detected faces with inverted colors. They debug the web rendering for an hour before realizing the image data was BGR and the browser expects RGB. The fix is one line, but only if you know where to put it.

The Coordinate System

This section covers the kind of detail that seems obvious until it costs you an afternoon. OpenCV's coordinate system puts the origin at the top-left corner of the image. The x-axis goes right, the y-axis goes down. This matches how screens work but confuses anyone coming from a math or plotting background where y increases upward.

Understanding the coordinate system is critical because every drawing function, every detection result, and every ROI extraction depends on it. When a face detector returns a bounding box at (x=120, y=80, w=100, h=130), that means 120 pixels from the left edge and 80 pixels down from the top. If you mentally flip the y-axis because you're thinking in Cartesian coordinates, your bounding boxes end up in the wrong part of the image.

# The pixel at (x=100, y=50) means:
# 100 pixels from the left edge
# 50 pixels from the top edge
# Access it as img[y, x] — note the order: row first, column second

pixel_value = img[50, 100]  # y=50, x=100

The key trap: when you call OpenCV drawing functions like cv2.rectangle(), coordinates are (x, y). But when you index the NumPy array directly, it's [y, x]. Two systems, two conventions, in the same library. Write a comment on day one, or write a bug on day two.

# Drawing uses (x, y)
cv2.rectangle(img, (100, 50), (200, 150), (0, 255, 0), 2)

# Array access uses [y, x]
region = img[50:150, 100:200]  # rows 50-150, columns 100-200

Reading and Displaying Your First Image

The basic pipeline: load an image, inspect it, display it (if you have a monitor), and save the result.

import cv2

# Load an image from disk
img = cv2.imread('/home/pi/photo.jpg')

# Check if it loaded (imread returns None on failure, doesn't raise)
if img is None:
    print("Failed to load image — check the file path")
    exit(1)

# Print basic info
h, w, channels = img.shape
print(f"Image: {w}x{h}, {channels} channels, dtype={img.dtype}")

# Display (only works with a monitor + opencv-python, not headless)
cv2.imshow('My Image', img)
cv2.waitKey(0)          # Wait for any keypress
cv2.destroyAllWindows() # Clean up the window

The waitKey() function is the heartbeat of any OpenCV GUI loop. waitKey(0) blocks forever until a keypress. waitKey(1) waits 1 millisecond — you'll use this in video loops to keep the display responsive. Without it, the window never actually renders. On a headless Pi, you skip imshow() and waitKey() entirely — your output goes to imwrite() or directly to a network stream. The GUI functions exist for development convenience, not for production use.

One detail worth noting: imread() supports different loading modes. By default, it loads a color image. You can load directly as grayscale by passing a flag:

# Load as color (default)
color_img = cv2.imread('/home/pi/photo.jpg', cv2.IMREAD_COLOR)

# Load directly as grayscale — skips the color → gray conversion step
gray_img = cv2.imread('/home/pi/photo.jpg', cv2.IMREAD_GRAYSCALE)
print(gray_img.shape)  # (480, 640) — 2D array, single channel

# Load with alpha channel preserved (if present)
alpha_img = cv2.imread('/home/pi/logo.png', cv2.IMREAD_UNCHANGED)

Loading directly as grayscale with IMREAD_GRAYSCALE is slightly faster than loading as color and then converting, because it skips the intermediate three-channel allocation. In a batch processing pipeline handling thousands of images, that difference adds up. For single-image operations the difference is negligible, but forming the habit of loading in the format you need — rather than loading the default and converting — is the kind of discipline that pays off on constrained hardware.

The Resolution Tax

Here's where the Pi's ARM chip forces you to think differently from a desktop developer.

I benchmarked a simple grayscale conversion on a Pi 4 (4 GB model):

import cv2
import time

def benchmark_resolution(img_path, target_width, target_height):
    img = cv2.imread(img_path)
    resized = cv2.resize(img, (target_width, target_height))

    start = time.perf_counter()
    for _ in range(100):
        gray = cv2.cvtColor(resized, cv2.COLOR_BGR2GRAY)
    elapsed = (time.perf_counter() - start) / 100

    print(f"{target_width}x{target_height}: {elapsed*1000:.2f} ms per frame")

# Results on Pi 4 (4 GB):
# 640x480:   ~0.8 ms per frame
# 1280x720:  ~2.4 ms per frame
# 1920x1080: ~5.1 ms per frame
# 3840x2160: ~19.7 ms per frame

At 640x480, you can afford complex processing pipelines and still hit real-time frame rates. At 4K, a single grayscale conversion takes 20 ms — and that's the cheapest operation in your pipeline. Add blurring, edge detection, and contour finding, and you're looking at 100+ ms per frame. That's 10 FPS before you've done anything useful.

The practical rule: capture at whatever resolution you need for accuracy, then immediately resize to the smallest resolution that still gives you correct results. For face detection, 640x480 is almost always sufficient. For reading text (OCR), you might need 1280x720. For counting large objects in a room, 320x240 works fine.

This isn't premature optimization — it's the foundational engineering constraint of computer vision on embedded hardware. Desktop developers can ignore resolution because their GPUs process images in parallel across thousands of cores. The Pi's ARM CPU processes pixels sequentially. Every pixel you don't process is a pixel you didn't have to wait for. I structure every project around this principle: determine the minimum resolution that produces correct results, then build the entire pipeline at that resolution. You can always increase resolution later if accuracy demands it. You can never get back the time you spent processing pixels that didn't contribute to the result.

What to Do Monday Morning