The Image Processing Pipeline

The trap is treating every image as ready to process the moment you load it. You grab a frame from the camera, pass it straight to your detection algorithm, and wonder why results are inconsistent. One frame detects a face perfectly; the next misses it entirely. The lighting changed. The resolution is wrong for the algorithm. The color space is BGR when the model expects grayscale. You're feeding garbage into a function and blaming the function.

Every serious computer vision system I build follows the same pattern: load, convert, resize, process, annotate, save. Six steps. In that order. Skip one and you're debugging symptoms instead of causes.

The processing pipeline isn't optional overhead — it's the engineering that makes everything downstream predictable. When your pipeline normalizes every frame to the same color space, the same resolution, and the same value range, your detection code doesn't need to handle variation. It handles one format, and it handles it well.

I learned this the hard way from medical imaging. In medicine, every diagnostic image goes through a standard preparation before a physician reads it — contrast adjustment, windowing, spatial normalization. No radiologist looks at raw sensor data and makes a diagnosis. The same discipline applies to computer vision. Your algorithm is only as reliable as the preparation that feeds it.

Color Space Conversions: Speaking the Right Language

OpenCV supports over 150 color space conversions through cv2.cvtColor(). You need three of them. The rest are for specialists in colorimetry and printing.

import cv2

img = cv2.imread('/home/pi/photo.jpg')

# BGR to Grayscale — the most common conversion
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
print(gray.shape)  # (480, 640) — notice: 2D, not 3D. Single channel.

# BGR to HSV — for color-based detection
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# BGR to RGB — for interoperability with other libraries
rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)

The grayscale conversion collapses three channels into one using a weighted formula: 0.299R + 0.587G + 0.114B. The weights match human luminance perception — green contributes the most because our eyes are most sensitive to it. This isn't an average; it's a perceptual model. That distinction matters when you're detecting objects by brightness contrast.

HSV — Hue, Saturation, Value — is the color space you reach for when you need to detect objects by color. BGR and RGB encode color as a mix of three primaries, which makes "find everything red" surprisingly hard. In HSV, hue is a single number representing the color angle on a wheel (0–180 in OpenCV, not 0–360), saturation is how vivid the color is, and value is brightness. Detecting a red ball becomes a range check on hue rather than a complex three-channel threshold.

import cv2
import numpy as np

img = cv2.imread('/home/pi/red_ball.jpg')
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# Red wraps around the hue wheel, so we need two ranges
lower_red1 = np.array([0, 120, 70])
upper_red1 = np.array([10, 255, 255])
lower_red2 = np.array([170, 120, 70])
upper_red2 = np.array([180, 255, 255])

mask1 = cv2.inRange(hsv, lower_red1, upper_red1)
mask2 = cv2.inRange(hsv, lower_red2, upper_red2)
red_mask = mask1 | mask2

# Apply the mask to the original image
result = cv2.bitwise_and(img, img, mask=red_mask)

Resizing: The Performance Lever

Resizing isn't cosmetic. On a Pi, it's the single most impactful optimization you can make. As we proved in the previous chapter, processing time scales linearly with pixel count. Resizing a 1080p frame to 480p before processing gives you a 4.5x speedup at the cost of some spatial detail.

The question isn't whether to resize — it's when and to what. I resize immediately after color conversion, before any processing. This means every operation downstream — blur, threshold, edge detection, contour finding — runs on the smaller image. If you resize at the end (for display), you've already paid the full computational cost of processing the large image. The savings come from resizing early, not late.

The target resolution depends on your task. A good heuristic: start at 640x480 and only increase if your detection accuracy suffers. Most face detection, motion detection, and object counting tasks work perfectly at 480p. OCR and small-text reading tasks sometimes need 720p. I've never needed more than 1080p for any Pi project in production.

import cv2

img = cv2.imread('/home/pi/large_photo.jpg')

# Resize to a specific dimension
small = cv2.resize(img, (640, 480))

# Resize by a scaling factor
half = cv2.resize(img, None, fx=0.5, fy=0.5)

# Resize with a specific interpolation method
high_quality = cv2.resize(img, (640, 480), interpolation=cv2.INTER_AREA)
fast_upscale = cv2.resize(img, (1280, 960), interpolation=cv2.INTER_LINEAR)

The interpolation method matters when you care about output quality:

• cv2.INTER_AREA — best for shrinking. Averages pixels in the source area. Use this as your default for downscaling.
• cv2.INTER_LINEAR — bilinear interpolation. Fast, decent quality. Good for upscaling when speed matters.
• cv2.INTER_CUBIC — bicubic interpolation. Better quality than linear, 2x slower. Use when the resized image is the final output.
• cv2.INTER_NEAREST — nearest neighbor. Fastest, worst quality. Produces blocky results. Only useful for masks and label maps where you need exact integer values preserved (bilinear interpolation would blur a mask's sharp edges and create gray pixels that break binary thresholding).

The Processing Pipeline Pattern

Every image processing script I write follows this skeleton. The order matters — each step normalizes the input for the next:

import cv2
import time

def process_image(input_path, output_path):
    # Step 1: Load
    img = cv2.imread(input_path)
    if img is None:
        raise FileNotFoundError(f"Cannot load: {input_path}")

    # Step 2: Convert color space
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

    # Step 3: Resize to processing resolution
    processed = cv2.resize(gray, (640, 480), interpolation=cv2.INTER_AREA)

    # Step 4: Process (example: Gaussian blur + edge detection)
    blurred = cv2.GaussianBlur(processed, (5, 5), 0)
    edges = cv2.Canny(blurred, 50, 150)

    # Step 5: Annotate (draw results on a copy of the original)
    annotated = img.copy()
    # ... draw bounding boxes, labels, etc. on annotated

    # Step 6: Save
    cv2.imwrite(output_path, annotated)
    return edges

result = process_image('/home/pi/input.jpg', '/home/pi/output.jpg')

The critical insight is step 5: annotate on a copy. Never draw on the image you're processing. If you draw a green rectangle on the frame and then run edge detection, the edge detector finds the rectangle's edges too. You've contaminated your data with your visualization.

Notice that steps 2 and 3 — convert and resize — happen on the result of step 1, not on the original. This means your processing pipeline operates on a normalized version of the input. The original image stays intact for annotation in step 5. Two parallel tracks: one for analysis (small, grayscale, processed) and one for display (full resolution, full color, annotated).

This separation sounds like overengineering until the first time you need to change your processing resolution without changing your display resolution, or switch from grayscale processing to HSV processing without touching the annotation code. Clean separation of concerns isn't a theoretical principle on the Pi — it's what keeps your code modifiable when requirements change.

Image Arithmetic and Masking

Images are NumPy arrays. That means you can add them, subtract them, blend them, and mask them with standard array operations. This is more powerful than it sounds.

import cv2
import numpy as np

img = cv2.imread('/home/pi/photo.jpg')

# Brighten an image by adding a constant
brighter = cv2.add(img, np.ones_like(img) * 50)

# Darken by subtracting
darker = cv2.subtract(img, np.ones_like(img) * 50)

# Blend two images (weighted addition)
overlay = cv2.imread('/home/pi/overlay.jpg')
overlay = cv2.resize(overlay, (img.shape[1], img.shape[0]))
blended = cv2.addWeighted(img, 0.7, overlay, 0.3, 0)

Masking is where arithmetic becomes useful for detection pipelines. A mask is a binary image — white (255) where you want to keep data, black (0) where you don't. Apply it with bitwise_and:

# Create a circular mask (e.g., to isolate the center of a frame)
mask = np.zeros(img.shape[:2], dtype=np.uint8)
h, w = img.shape[:2]
cv2.circle(mask, (w // 2, h // 2), min(h, w) // 3, 255, -1)

# Apply the mask
masked = cv2.bitwise_and(img, img, mask=mask)

This pattern — create a mask from detection results, then apply it to extract or highlight regions — is the backbone of every color segmentation and object isolation pipeline.

One more arithmetic operation worth knowing: background subtraction. If you have a static reference frame (the scene with nothing in it), you can subtract it from the current frame to isolate anything that moved or appeared:

# Capture a reference frame (empty scene)
reference = cv2.imread('/home/pi/empty_room.jpg')
reference_gray = cv2.cvtColor(reference, cv2.COLOR_BGR2GRAY)
reference_gray = cv2.GaussianBlur(reference_gray, (21, 21), 0)

# Current frame
current = cv2.imread('/home/pi/room_with_person.jpg')
current_gray = cv2.cvtColor(current, cv2.COLOR_BGR2GRAY)
current_gray = cv2.GaussianBlur(current_gray, (21, 21), 0)

# Absolute difference
diff = cv2.absdiff(reference_gray, current_gray)

# Threshold to create a binary mask
_, motion_mask = cv2.threshold(diff, 25, 255, cv2.THRESH_BINARY)

This is the simplest motion detection algorithm — and it works surprisingly well on a Pi for monitoring doorways, counting visitors, or detecting packages on a porch. The Gaussian blur before the subtraction reduces noise sensitivity, and the threshold converts gradual changes (shadows, lighting shifts) into a clean binary mask where only significant movement appears white.

Saving Images and Format Considerations

cv2.imwrite() handles saving. The format is determined by the file extension:

import cv2

img = cv2.imread('/home/pi/photo.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# JPEG — lossy, small file, good for photos
cv2.imwrite('/home/pi/output.jpg', gray)

# PNG — lossless, larger file, good for processed results
cv2.imwrite('/home/pi/output.png', gray)

# Control JPEG quality (0-100, default 95)
cv2.imwrite('/home/pi/compressed.jpg', gray, [cv2.IMWRITE_JPEG_QUALITY, 70])

# Control PNG compression (0-9, higher = smaller but slower)
cv2.imwrite('/home/pi/compressed.png', gray, [cv2.IMWRITE_PNG_COMPRESSION, 9])

On a Pi with limited SD card space, JPEG at quality 70 gives you roughly 80% of the visual quality at 30% of the file size compared to quality 95. For processed results where exact pixel values matter — masks, edge maps, annotated frames — use PNG. For everything else, JPEG.

There's a subtlety here that trips up batch processing pipelines. imwrite() returns True on success and False on failure. Like imread(), it doesn't raise exceptions. If the target directory doesn't exist, it silently returns False and your script continues running without saving anything. Always check the return value, or at minimum verify the file exists after writing:

import cv2
import os

success = cv2.imwrite('/home/pi/output/result.jpg', img)
if not success:
    print("Failed to save — does the output directory exist?")

# Or verify after the fact
assert os.path.exists('/home/pi/output/result.jpg'), "Save failed"

I build every pipeline with a dedicated output directory created at startup. The two minutes you spend on os.makedirs(output_dir, exist_ok=True) at the top of your script saves you from discovering at 3 AM that your overnight processing run saved zero frames because the directory was missing.

For batch processing on a Pi, file I/O is another bottleneck worth watching. Writing a JPEG at quality 95 takes roughly 15 ms on a Pi 4 — fast enough for a single image, but problematic in a 30 FPS video pipeline where you're saving every frame. At that rate, imwrite alone consumes half your frame budget. The solution is either reducing save frequency (save every 10th frame instead of every frame) or dropping JPEG quality to 60–70, which cuts write time roughly in half. For most monitoring applications, you don't need every frame — you need every interesting frame, and your detection code decides which frames qualify.

What to Do Monday Morning