Skip to lesson
Westlake RoboticsLearn
Vault

FIRST Tech Challenge

Loading progress…

  • VisionPortal Camera Setup
  • OpenCV Color and Region Processors
  • AprilTags and Field Pose
  • Vision Fallbacks and Confidence

OpenCV Color and Region Processors

Turn pixels into LEFT/CENTER/RIGHT decisions.

Vision and AprilTagsVision

In this lesson, you will:

  • Choose a color space.
  • Measure regions.
  • Expose a simple enum.

Concept narrative

A color pipeline reduces an image to a decision. The processor can know about rectangles and channels, but autonomous should receive a simple result and confidence.

Robot mental model

The camera sees pixels; the robot needs a plan. Regions of interest are the bridge between raw image data and a field-specific decision.

Your browser does not support embedded video. Use the open-video link below.
0:00 / 0:00
Vision decisions: gates, rejection reasons, and fallbackOpen video Download captions
Video transcriptRead or search the narration

You already know that a vision processor gives the program an observation, not a motor command. Let's make the decision after the observation explicit. It either passes every declared gate, or it becomes UNKNOWN with a reason and uses the team's conservative fallback. Our region processor and an AprilTag processor do not expose the same measurements. They should not pretend to share one universal confidence number. A region result can check its strongest score, the margin over the second-best score, and stability. An AprilTag result can check that the detection and metadata exist, that the ID is allowed, and that range and bearing are inside reviewed bounds. It can also check that the observation is fresh and that the same ID survives for three consecutive accepted frames. The range, bearing, age, and score limits are example numbers used to trace the decision. The team must choose and validate its real policy from labelled evidence, such as the tag's apparent size, distance, and reprojection error. What matters here is that every gate is visible and every failure has a name. The Decision object keeps three things together: the observed result, the branch the autonomous code will use, and the acceptance or rejection reason. The region-score policy rejects a weak winner, a low-margin winner, or an unstable result. The AprilTag policy rejects no detection, missing metadata, or a disallowed tag ID before its later gates can pass. After those checks pass, it counts consecutive frames for the same ID. The first two accepted frames still return UNSTABLE, but the third can select an actual branch. Any failed gate or ID change resets that count. The FTC AprilTag API supplies detections and pose fields such as range and bearing when metadata is available. Observation age and the limits in this example are team-defined policy, not a universal SDK confidence field. Back in the overall view, a weak region becomes TOO_WEAK. Two close regions become LOW_MARGIN. AprilTag failures have names such as NO_DETECTION, MISSING_METADATA, DISALLOWED_ID, OUT_OF_BOUNDS, and STALE. A candidate that has not yet survived enough frames is UNSTABLE. Only an observation that passes the whole policy with no failed gates becomes ACCEPTED. These names are more useful than a single false value because they tell us which assumption failed. They also prevent an old accepted branch from silently surviving after the newest observation was rejected. The branch table stays deliberately small. An accepted observation selects left, center, or right. Anything rejected produces the result UNKNOWN and selects CENTER_SAFE. CENTER_SAFE in this example fixture is not proof that center is safe for every season. It is only an example of what autonomous branch would run in a fallback mode. The team must choose the real fallback. Replay labelled observations with robot outputs held at zero. Record the first failed gate, result, reason, selected branch, and proof that rejected data never requests motion. Only after the policy passes should the actual camera and robot be validated. Vision becomes more complicated when hardware is involved, so use the proper precautions. A reliable system needs many checks, testing, and tuning; try the policy against real evidence and keep improving it.

Implementation walkthrough

Convert color space, crop regions, calculate scores, draw rectangles, and expose getPosition. Print raw scores before thresholding.

PropProcessor.javaJava

double leftScore = Core.mean(ycrcb.submat(leftRect)).val[1];
double centerScore = Core.mean(ycrcb.submat(centerRect)).val[1];
double rightScore = Core.mean(ycrcb.submat(rightRect)).val[1];

double best = Math.max(leftScore, Math.max(centerScore, rightScore));
double second = leftScore + centerScore + rightScore - best
    - Math.min(leftScore, Math.min(centerScore, rightScore));

if (best - second < minimumMargin) {
    position = PropPosition.UNKNOWN;
} else if (best == leftScore) {
    position = PropPosition.LEFT;
} else if (best == centerScore) {
    position = PropPosition.CENTER;
} else {
    position = PropPosition.RIGHT;
}

Common mistakes and debugging

Thresholds fail under lighting changes. If a decision is wrong, inspect the image and raw scores before changing autonomous branches.

Practice

Build a two-region detector and test it under at least two lighting conditions.

Checkpoint

  • LEFT, CENTER, and RIGHT regions are drawn and scored.
  • Raw scores and the winning margin are visible.
  • Low-margin frames return UNKNOWN instead of a confident branch.
  • The test record includes the setup, prediction, and observed result.
  • A teammate can repeat the check from the saved evidence without guessing.

Reflection check

Check your understanding before moving on.

Why should a three-region OpenCV processor return UNKNOWN when the winning margin is small?
What should be inspected before changing a color threshold after a wrong branch?

0 of 2 answered

References

FTC VisionPortal DocsOfficial FTC camera and processor lifecycle documentation.GM0 Computer VisionFTC-oriented vision concepts, pipelines, and practical advice.FTC AprilTag DocsOfficial AprilTag detection and metadata reference.
Loading lesson progress
Previous lessonVisionPortal Camera SetupNext lessonAprilTags and Field Pose