Exposure for indoors #86

jgoppert · 2016-10-12T05:34:03Z

@LorenzMeier @bkueng @ecmnet @mhkabir please test, this is a huge improvement for indoors and I suspect outdoors as well.

Changes

Add a lot more control over camera gain and exposure, can set with params not to allow fast tuning
Tune exposure and gain settings to work well with low light and outdoor/ bright light settings using auto gain and exposure control.
Instead of publishing flow every 10 received frames, publishes at fixed rate set by PUB_RATE param

ecmnet · 2016-10-14T11:50:07Z

@jgoppert I've not setup my development environment for PX4Flow. Could you provide a binary to upload?

jgoppert · 2016-10-14T12:23:28Z

@ecmnet, I've uploaded the built firmware to my google drive here: https://drive.google.com/open?id=0B3wIEs1gcVahNktlQkhCMndIeUE

ecmnet · 2016-10-14T12:26:11Z

@jgoppert Thanks a lot! I'll give it a try a soon as I've mounted flow to my device again.

ecmnet · 2016-10-14T13:08:31Z

@jgoppert Unfortunately, my PX4Flow does not work after upgrading (does not connect to QGC at all)

jgoppert · 2016-10-14T14:52:04Z

Weird, will try uploading the image I created for you tonight on my device.

ecmnet · 2016-10-15T07:20:03Z

@jgoppert I've setup my environment now and was able to compile and upload this branch. Is it correct to have some white square dots in the image now?

jgoppert · 2016-10-15T15:02:56Z

Yes, the dots indicate high quality optical flow in that region of the image.

jgoppert · 2016-10-15T15:08:28Z

Or more specifically, that 8x8 tile had enough difference between its pixels (f thrsh param) that it included that tile in the optical flow calculation.

ecmnet · 2016-10-15T17:07:48Z

@jgoppert Thanks for clarification. I will do some testflights as soon I've mounted flow again.

ecmnet · 2016-11-21T11:36:58Z

@jgoppert Flow quality seems not to be reasonable with this patch. Even, if I darken the lens, quality is reported at 220-250 in my case.

jgoppert · 2016-11-21T15:14:25Z

It increases the gain and exposure when you cover the lens so you start picking up noise. Look at the image in qgc. The max gain and max exposure can be lowered to limit the effect, but if you don't cover the lens I think it gives good low and high light performance.

ChristophTobler

Looks good. I think I can give this a test the next few days

ChristophTobler · 2017-03-22T16:25:33Z

src/modules/flow/main.c

@@ -699,7 +707,7 @@ int main(void)
 			uint16_t frame = 0;
 			for (frame = 0; frame < image_size_send / MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN + 1; frame++)
 			{
-				mavlink_msg_encapsulated_data_send(MAVLINK_COMM_2, frame, &((uint8_t *) current_image)[frame * MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN]);
+				mavlink_msg_encapsulated_data_send(MAVLINK_COMM_2, frame, &((uint8_t *) previous_image)[frame * MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN]);


Why did you change this to previous_image?

so that it gets overlayed with the dots for flow quality

ChristophTobler · 2017-03-22T16:26:52Z

src/modules/flow/mt9v034.c

@@ -79,7 +79,7 @@ void mt9v034_context_configuration(void)
 	uint16_t new_height_context_b = FULL_IMAGE_COLUMN_SIZE * 4;

 	/* blanking settings */
-	uint16_t new_hor_blanking_context_a = 709 + MINIMUM_HORIZONTAL_BLANKING;// 709 is minimum value without distortions
+	uint16_t new_hor_blanking_context_a = 425 + MINIMUM_HORIZONTAL_BLANKING;// 709 is minimum value without distortions


comment needs to be adapted

lericson · 2017-06-16T13:44:20Z

I've tested this and it doesn't work very well at all for me. It might be my hardware of course but I get pretty heavy banding with this.

jgoppert · 2017-06-16T15:23:04Z

@lericson Go into qgc and play with exposure_max and brightness. Lowering brightness down should fix it for you (try brightness 10, max_exposure 300). I've tested this a lot, so trust me, it is a huge difference for indoors. But watch out for specular reflections and shadows. I've seen the banding before as well, it is also there on the defualt firmware. It just can be more obvious when the auto gain is adjusting etc.

lericson · 2017-06-16T15:49:02Z

@jgoppert I played around a bit with the parameters. Couldn't get very good results sadly. For your perusing, here is my merged version of this PR and #90: https://github.com/KTH-CAS-UAV/Flow/tree/master-with-exposure-pr

jgoppert · 2017-06-16T15:54:55Z

Don't have time to load it myself. What does your flow quality say? It should be much higher with this patch. Also, when you evaluate the quality, don't just look at the image, enable the flag SEND_LPOS. This will give you output of a system onboard estimator of the position, assuming that the board is always aligned with north. Then I usually walk in a 2 m square.

lericson · 2017-06-17T17:07:51Z

It said 255 all the time and the image quality was not very good with heavy banding. The flow estimation was also very noisy, which isn't so surprising. 🙂 - Ludvig

…

On 16 Jun 2017, at 18:14, James Goppert ***@***.***> wrote: @jgoppert commented on this pull request. In src/modules/flow/main.c: > @@ -699,7 +707,7 @@ int main(void) uint16_t frame = 0; for (frame = 0; frame < image_size_send / MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN + 1; frame++) { - mavlink_msg_encapsulated_data_send(MAVLINK_COMM_2, frame, &((uint8_t *) current_image)[frame * MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN]); + mavlink_msg_encapsulated_data_send(MAVLINK_COMM_2, frame, &((uint8_t *) previous_image)[frame * MAVLINK_MSG_ENCAPSULATED_DATA_FIELD_DATA_LEN]); so that it gets overlayed with the dots for flow quality — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

jgoppert · 2017-06-17T17:36:17Z

When you set the parameters it doesn't stick after a reboot, if you set brightness to 8 and max exposure to 200 (in the firmware and rebulid), that is bascially the same as the default firmware. This firmware is basically just giving you more control over the settings.

Brightness is basically the desired mean of the pixel values and it plays with auto gain to and auto exposure to get that.

It also shows you the debug for the optical flow, the dots mean that the algorithm is using that cell for flow calculation. If you turn it really dark and you see no dots, there is not enough difference in intensity between the pixels in that cell for it to use it for the flow calculation.

It also stabilizes the flow rate output to the rate you set instead of depending on your exposure setting.

jgoppert · 2017-06-17T17:41:11Z

makefiles/baremetal/toolchain_gnu-arm-eabi.mk

@@ -50,7 +50,7 @@ OBJDUMP			 = $(CROSSDEV)objdump

 # Check if the right version of the toolchain is available
 #
-CROSSDEV_VER_SUPPORTED	 = 4.7.4 4.7.5 4.7.6 4.8.4 4.9.3
+CROSSDEV_VER_SUPPORTED	 = 4.7.4 4.7.5 4.7.6 4.8.4 4.9.3 5.4.1


This supports the latest gcc, so I included it here.

jgoppert · 2017-06-17T17:41:48Z

src/include/settings.h

@@ -40,7 +40,7 @@
 #define ONBOARD_PARAM_NAME_LENGTH 		15
 #define BOTTOM_FLOW_IMAGE_HEIGHT		64
 #define BOTTOM_FLOW_IMAGE_WIDTH			64
-#define BOTTOM_FLOW_SEARCH_WINDOW_SIZE 	4
+#define BOTTOM_FLOW_SEARCH_WINDOW_SIZE 	8


You can play with this, but this makes it possible for it to see a large max velocity, but you have to watch the update rate and make sure the cpu isn't maxing out.

jgoppert · 2017-06-17T17:42:09Z

src/include/settings.h

+	PARAM_HDR,
+	PARAM_AEC,
+	PARAM_AGC,
+	PARAM_BRIGHT,


Here I exposed settings that were hard coded in the firmware.

jgoppert · 2017-06-17T17:42:32Z

src/modules/flow/communication.c

+								dma_reconfigure();
+								buffer_reset();
+							}
+


This is just setting the parameters that you request in QGC.

jgoppert · 2017-06-17T17:43:01Z

src/modules/flow/flow.c

+
+			}
+		}
+	}


This piece of code was commented out and shows you flow quality indicators for each grid cell.

jgoppert · 2017-06-17T17:43:50Z

src/modules/flow/main.c

-			if (counter % (uint32_t)global_data.param[PARAM_BOTTOM_FLOW_SERIAL_THROTTLE_FACTOR] == 0)//throttling factor
+			uint32_t now = get_boot_time_us();
+			uint32_t time_since_last_pub = now - time_last_pub;
+			if (time_since_last_pub > (1.0e6f/global_data.param[PARAM_BOTTOM_FLOW_PUB_RATE]))


This is the logic to publish at a fixed rate instead of based on the exposure setting.

jgoppert · 2017-06-17T17:44:35Z

src/modules/flow/main.c

-					lpos.vz = 0;
+					lpos.vx = ground_distance*accumulated_flow_x/integration_timespan;
+					lpos.vy = ground_distance*accumulated_flow_y/integration_timespan;
+					lpos.vz = 0; // no direct measurement, just ignore


I included the velocity here for debug.

jgoppert · 2017-06-17T17:45:53Z

src/modules/flow/settings.c

+	strcpy(global_data.param_name[PARAM_AGC], "AGC");
+	global_data.param_access[PARAM_AGC] = READ_WRITE;
+
+	global_data.param[PARAM_BRIGHT] = 20;


If you have trouble with washout/ banding, try lowering this value. 8 is the minimum.

jgoppert · 2017-06-17T17:46:46Z

src/modules/flow/settings.c

+	strcpy(global_data.param_name[PARAM_BOTTOM_FLOW_PUB_RATE], "BFLOW_RATE");
+	global_data.param_access[PARAM_BOTTOM_FLOW_PUB_RATE] = READ_WRITE;
+
+	global_data.param[PARAM_EXPOSURE_MAX] = 500;


Increasing this value helps in low light, but it also makes the max velocity you can go lower. If you have trouble with velocity maxing out, lower this to around 200 or 100.

jgoppert · 2017-06-17T17:47:38Z

src/modules/flow/settings.c

+	strcpy(global_data.param_name[PARAM_EXPOSURE_MAX], "EXPOSURE_MAX");
+	global_data.param_access[PARAM_EXPOSURE_MAX] = READ_WRITE;
+
+	global_data.param[PARAM_GAIN_MAX] = 16;


Gain can make the image pixelated. If your image has a lot of noise, try lowering this to 4 or 8.

jgoppert · 2017-06-17T17:49:40Z

@lericson I tried to explain my changes in this patch, hope that helps and you find some of it useful for your work.

…e time

ChristophTobler · 2017-08-15T15:17:21Z

@jgoppert I have bench tested this PR so far. Looks very good. I increased the flow feature threshold because it reported good quality even though it was just noisy and pointing to a plain white surface. I think it's safer this way.

jgoppert force-pushed the exposure_for_indoors branch from 5ed0082 to 08ced78 Compare October 12, 2016 05:36

Improve performance for indoors and auto exposure/gain, add more params.

ee88862

jgoppert force-pushed the exposure_for_indoors branch from 08ced78 to ee88862 Compare October 12, 2016 05:41

Allow gcc 5.4.1

66c6be9

jgoppert mentioned this pull request Jan 11, 2017

EKF2 flow integration_timespan check is invalid PX4/PX4-Autopilot#6282

Closed

Lower exposure max and increase desired brightness.

1b2904f

ChristophTobler suggested changes Mar 22, 2017

View reviewed changes

jgoppert commented Jun 17, 2017

View reviewed changes

src/modules/flow/flow.c

}

}

}

Copy link

Member Author

jgoppert Jun 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This piece of code was commented out and shows you flow quality indicators for each grid cell.

jgoppert commented Jun 17, 2017

View reviewed changes

ChristophTobler mentioned this pull request Aug 15, 2017

Improve flow quality #90

Open

increase flow feature threshold to avoid having a high quality all th…

0f8f974

…e time

ChristophTobler approved these changes Aug 15, 2017

View reviewed changes

LorenzMeier merged commit 9eb8fb3 into master Aug 15, 2017

ChristophTobler deleted the exposure_for_indoors branch August 16, 2017 07:05

juancamilog mentioned this pull request Sep 3, 2018

Bumping the firmware to newer changes on PX4/Flow repo priseborough/px4flow#4

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exposure for indoors #86

Exposure for indoors #86

jgoppert commented Oct 12, 2016 •

edited

Loading

ecmnet commented Oct 14, 2016

jgoppert commented Oct 14, 2016 •

edited

Loading

ecmnet commented Oct 14, 2016

ecmnet commented Oct 14, 2016

jgoppert commented Oct 14, 2016

ecmnet commented Oct 15, 2016

jgoppert commented Oct 15, 2016

jgoppert commented Oct 15, 2016

ecmnet commented Oct 15, 2016

ecmnet commented Nov 21, 2016

jgoppert commented Nov 21, 2016

ChristophTobler left a comment

ChristophTobler Mar 22, 2017

jgoppert Jun 16, 2017

ChristophTobler Mar 22, 2017

lericson commented Jun 16, 2017

jgoppert commented Jun 16, 2017 •

edited

Loading

lericson commented Jun 16, 2017

jgoppert commented Jun 16, 2017 •

edited

Loading

lericson commented Jun 17, 2017 via email

jgoppert commented Jun 17, 2017 •

edited

Loading

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert Jun 17, 2017

jgoppert commented Jun 17, 2017

ChristophTobler commented Aug 15, 2017

Exposure for indoors #86

Exposure for indoors #86

Conversation

jgoppert commented Oct 12, 2016 • edited Loading

Changes

ecmnet commented Oct 14, 2016

jgoppert commented Oct 14, 2016 • edited Loading

ecmnet commented Oct 14, 2016

ecmnet commented Oct 14, 2016

jgoppert commented Oct 14, 2016

ecmnet commented Oct 15, 2016

jgoppert commented Oct 15, 2016

jgoppert commented Oct 15, 2016

ecmnet commented Oct 15, 2016

ecmnet commented Nov 21, 2016

jgoppert commented Nov 21, 2016

ChristophTobler left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lericson commented Jun 16, 2017

jgoppert commented Jun 16, 2017 • edited Loading

lericson commented Jun 16, 2017

jgoppert commented Jun 16, 2017 • edited Loading

lericson commented Jun 17, 2017 via email

jgoppert commented Jun 17, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jgoppert commented Jun 17, 2017

ChristophTobler commented Aug 15, 2017

jgoppert commented Oct 12, 2016 •

edited

Loading

jgoppert commented Oct 14, 2016 •

edited

Loading

jgoppert commented Jun 16, 2017 •

edited

Loading

jgoppert commented Jun 16, 2017 •

edited

Loading

jgoppert commented Jun 17, 2017 •

edited

Loading