|
| 1 | +AprilTag Challenges in DECODE presented by RTX |
| 2 | +============================================== |
| 3 | + |
| 4 | +What are AprilTags? |
| 5 | +------------------- |
| 6 | + |
| 7 | +Developed at the `University of Michigan |
| 8 | +<https://april.eecs.umich.edu/software/apriltag>`_, AprilTags are similar to a |
| 9 | +2D barcode or a simplified QR Code. It contains a numeric **ID code** and can |
| 10 | +be used for **location and orientation**. |
| 11 | + |
| 12 | +In *FIRST* Tech Challenge during the DECODE presented by RTX season, AprilTags |
| 13 | +are used in three different ways: |
| 14 | + |
| 15 | +1. On the OBELISK, the AprilTags are used to identify one of three MOTIFS that |
| 16 | + are randomized each MATCH. |
| 17 | +2. On the GOALS, AprilTags can be used to target the GOAL for teams to launch |
| 18 | + ARTIFACTS accurately into the correct GOAL. |
| 19 | +3. On the GOALS, AprilTags can be used as a visual odometry system, using the |
| 20 | + information that AprilTags can provide to calculate the position of the |
| 21 | + ROBOT on the FIELD (through a process called localization). See the |
| 22 | + :doc:`AprilTag Localization <../../vision_portal/apriltag_localization/apriltag-localization>` |
| 23 | + page for more information. |
| 24 | + |
| 25 | +.. figure:: images/decode-apriltags.png |
| 26 | + :width: 50% |
| 27 | + :align: center |
| 28 | + :alt: Image showing the DECODE field and AprilTag locations |
| 29 | + |
| 30 | + AprilTag IDs and Locations on the DECODE field. |
| 31 | + |
| 32 | +AprilTags with Difficult Environmental Lighting |
| 33 | +----------------------------------------------- |
| 34 | + |
| 35 | +One of the challenges teams will face this season is ensuring that their cameras |
| 36 | +are able to see the AprilTags correctly. AprilTags rely on the fact that the |
| 37 | +white and black portions of the AprilTags are contrasting colors - if the lighting |
| 38 | +in the environment doesn't allow for enough contrast, the AprilTag algorithm |
| 39 | +may not properly detect the AprilTag. Fortunately, there are things we can do |
| 40 | +with virtually every webcam that can help correct for environmental issues. |
| 41 | + |
| 42 | +An excellent example situation came up in a warehouse. The DECODE field was set |
| 43 | +up in the warehouse, and using default settings with the |
| 44 | +``ConceptAprilTagEasy`` sample. When viewing the camera stream preview, the |
| 45 | +AprilTag on the OBELISK was completely washed out by sunlight striking the |
| 46 | +OBELISK directly on a sunny day, making the AprilTag unable to be seen. A |
| 47 | +different camera at a slightly different angle took another picture of the same |
| 48 | +scene, and the AprilTag can be seen but there is definitely too much direct light |
| 49 | +reflecting off the AprilTag, making it unable to be recognized. This scenario is |
| 50 | +very similar to a gymnasium where an event could be hosted, and on a sunny day |
| 51 | +light can interfere with a camera's ability to view an AprilTag. What can be done? |
| 52 | + |
| 53 | +.. only:: html |
| 54 | + |
| 55 | + .. grid:: 1 2 2 3 |
| 56 | + :gutter: 2 |
| 57 | + |
| 58 | + .. grid-item-card:: |
| 59 | + :class-header: sd-bg-dark font-weight-bold sd-text-white |
| 60 | + :class-body: sd-text-left body |
| 61 | + |
| 62 | + Image #1 - Example |
| 63 | + |
| 64 | + ^^^ |
| 65 | + |
| 66 | + .. figure:: images/1-decode-washed-out-obelisk.* |
| 67 | + :align: center |
| 68 | + :width: 95% |
| 69 | + :alt: Image of DECODE field with obelisk AprilTag unable to be seen |
| 70 | + |
| 71 | + +++ |
| 72 | + |
| 73 | + Washed Out AprilTag on OBELISK |
| 74 | + |
| 75 | + .. grid-item-card:: |
| 76 | + :class-header: sd-bg-dark font-weight-bold sd-text-white |
| 77 | + :class-body: sd-text-left body |
| 78 | + |
| 79 | + Image #2 - Alternate View |
| 80 | + |
| 81 | + ^^^ |
| 82 | + |
| 83 | + .. figure:: images/2-decode-washed-out-obelisk.* |
| 84 | + :align: center |
| 85 | + :width: 85% |
| 86 | + :alt: Image of DECODE field from another perspective |
| 87 | + |
| 88 | + +++ |
| 89 | + |
| 90 | + Alternate View of OBELISK |
| 91 | + |
| 92 | + .. grid-item-card:: |
| 93 | + :class-header: sd-bg-dark font-weight-bold sd-text-white |
| 94 | + :class-body: sd-text-left body |
| 95 | + |
| 96 | + Image #3 - Alternate View |
| 97 | + |
| 98 | + ^^^ |
| 99 | + |
| 100 | + .. figure:: images/5-decode-warehouse-lighting.* |
| 101 | + :align: center |
| 102 | + :width: 85% |
| 103 | + :alt: Image showing light coming in from windows of warehouse |
| 104 | + |
| 105 | + +++ |
| 106 | + |
| 107 | + Sunlight Entering Warehouse |
| 108 | + |
| 109 | +.. only:: latex |
| 110 | + |
| 111 | + .. list-table:: Different Views of Challenging Scenario |
| 112 | + :class: borderless |
| 113 | + |
| 114 | + * - .. image:: images/1-decode-washed-out-obelisk.* |
| 115 | + - .. image:: images/2-decode-washed-out-obelisk.* |
| 116 | + - .. image:: images/5-decode-warehouse-lighting.* |
| 117 | + |
| 118 | +The best way to counter this environmental lighting is to use the webcam |
| 119 | +settings within the SDK to adjust both the Gain and the Exposure settings at |
| 120 | +the same time. By simultaneously minimizing the exposure (lessening the amount of |
| 121 | +time light is allowed to strike the sensor each image frame) and maximizing |
| 122 | +the gain (amplifying the signal from the sensor) the resulting image will be |
| 123 | +darker than a normal image but elements of high contrast will be accentuated, |
| 124 | +like AprilTags, allowing them to be recognized. This can be experimented with |
| 125 | +using the ``ConceptAprilTagOptimizeExposure`` sample. |
| 126 | + |
| 127 | +Sure enough, by minimizing the Exposure and maximizing the Gain of the webcam, |
| 128 | +the resulting images from the webcam were able to be used to recognize the |
| 129 | +problematic AprilTags. For more examples, the ``RobotAutoDriveToAprilTag...`` |
| 130 | +sample OpModes also use this technique for adjusting the exposure and gain |
| 131 | +settings of the camera to ensure the AprilTags are readable under most |
| 132 | +conditions. |
| 133 | + |
| 134 | +.. tip:: |
| 135 | + One big advantage is that this technique (minimizing exposure while |
| 136 | + maximizing gain) is ALSO very popular in reducing motion blur for reading |
| 137 | + AprilTags while moving - so this has more than one benefit! |
| 138 | + |
| 139 | +Here are examples of the images once the exposure and gain are set appropriately, |
| 140 | +one image has the AprilTag processing enabled to show that the AprilTag is |
| 141 | +being detected properly, and the other has processing disabled so that we can |
| 142 | +see the raw image being returned by the webcam. |
| 143 | + |
| 144 | +.. only:: html |
| 145 | + |
| 146 | + .. grid:: 1 2 2 2 |
| 147 | + :gutter: 2 |
| 148 | + |
| 149 | + .. grid-item-card:: |
| 150 | + :class-header: sd-bg-dark font-weight-bold sd-text-white |
| 151 | + :class-body: sd-text-left body |
| 152 | + |
| 153 | + Image #4 - Processed Image |
| 154 | + |
| 155 | + ^^^ |
| 156 | + |
| 157 | + .. figure:: images/3-decode-recognized-obelisk.* |
| 158 | + :align: center |
| 159 | + :width: 95% |
| 160 | + :alt: Image of DECODE field with obelisk AprilTag being processed |
| 161 | + |
| 162 | + +++ |
| 163 | + |
| 164 | + Processed Image showing Detections |
| 165 | + |
| 166 | + .. grid-item-card:: |
| 167 | + :class-header: sd-bg-dark font-weight-bold sd-text-white |
| 168 | + :class-body: sd-text-left body |
| 169 | + |
| 170 | + Image #5 - Raw Processed Image |
| 171 | + |
| 172 | + ^^^ |
| 173 | + |
| 174 | + .. figure:: images/4-decode-recognized-obelisk-raw.* |
| 175 | + :align: center |
| 176 | + :width: 95% |
| 177 | + :alt: Image of raw processed DECODE field |
| 178 | + |
| 179 | + +++ |
| 180 | + |
| 181 | + Image without AprilTag processing |
| 182 | + |
| 183 | +.. only:: latex |
| 184 | + |
| 185 | + .. list-table:: Resulting Images |
| 186 | + :class: borderless |
| 187 | + |
| 188 | + * - .. image:: images/3-decode-recognized-obelisk.* |
| 189 | + - .. image:: images/4-decode-recognized-obelisk-raw.* |
| 190 | + |
| 191 | +Reading Multiple AprilTags on the OBELISK |
| 192 | +----------------------------------------- |
| 193 | + |
| 194 | +The OBELISK is an equilateral triangular prism (we know, real obelisks have 4 |
| 195 | +sides) which is positioned with 1 of the rectangular faces centered on the |
| 196 | +GOAL-side of the FIELD, just outside of the FIELD perimeter. When ROBOTS are |
| 197 | +set up on the field contacting their ALLIANCE'S GOAL, it is a very real |
| 198 | +possibility that the ROBOT's camera will see and process multiple AprilTags. |
| 199 | + |
| 200 | +.. warning:: |
| 201 | + It might seem logical to read both AprilTags and use those two tags to |
| 202 | + determine (and verify) which AprilTag is actually being seen. However, there |
| 203 | + is no defined order for AprilTags on an OBELISK, so this is not reliable. |
| 204 | + |
| 205 | + |
| 206 | +.. figure:: images/6-decode-obelisk-tags.* |
| 207 | + :align: center |
| 208 | + :width: 75% |
| 209 | + :alt: Image showing OBELISK with more than one AprilTag visible |
| 210 | + |
| 211 | + View of AprilTags on OBELISK from BLUE GOAL |
| 212 | + |
| 213 | +A reliable way to determine which AprilTag is truly showing on the FIELD |
| 214 | +is to move the ROBOT into a position where the AprilTag on the front face of |
| 215 | +the OBELISK is the only tag that can be viewed. |
| 216 | + |
| 217 | +Good Luck this season! |
0 commit comments