adding video compression (h264) #44

bsobhani · 2023-12-13T17:10:13Z

Starting pull request for video compression (h264) support. Not close to being perfect, but getting the process started so that it does not linger.

Basic functionality works and seems stable on the machines I tested it on. Achieves lower compression sizes with higher quality than existing codecs for some types of image sequences. For example, if the differences between frames is consistently small, like at SNS beamlines where neutron events accumulate on an image.

To compile, set:

WITH_VC=YES
VC_EXTERNAL=NO

To use, select "H264" from $(P)$(R)Compressor.

Tuning PVs:

$(P)$(R)QMinMax - 1 for least lossless compression, higher values for lossier compression
$(P)$(R)GOPSize - Group of picture size. 1 for frame-by-frame compression. Set to higher values for true video compression.

ericonr

The code and Makefiles still have a lot of comments and debugging printfs...

ericonr · 2024-03-06T14:10:38Z

supportApp/videoCompressionSrc/README

+Prerequisites:
+* ffmpeg needs to be installed. "yum install ffmpeg-devel"
+
+PVs:
+* $(P)$(R)QMinMax - 1 for least lossless compression, higher values for lossier compression
+* $(P)$(R)GOPSize - Group of picture size. 1 for frame-by-frame compression. Set to higher values for true video compression.
+


Doesn't this PV documentation belong in NDPluginCodec?

And "1 for least lossy compression"

ericonr · 2024-03-06T14:12:03Z

supportApp/videoCompressionSrc/codec_tools.c

+AVCodecContext *c_c = 0; 
+int first = 1;
+AVCodecContext *c_d = 0; 
+int count_d = 0;
+
+
+const AVCodec *codec;


Wouldn't it be best to avoid global variables? Doesn't this mean a single IOC can only have a single compressed stream?

ericonr · 2024-03-06T14:13:23Z

supportApp/videoCompressionSrc/codec_tools.c

+	c->bit_rate = 400000;
+	c->width = width;
+	c->height = height;
+	//c->qmin = 1;
+	//c->qmax = 20;
+	c->time_base = (AVRational){1, 25};
+	c->framerate = (AVRational){25, 1};
+	//av_opt_set(c->priv_data, "crf", "1", AV_OPT_SEARCH_CHILDREN);
+	//av_opt_set(c->priv_data, "crf", "0", 0);
+	//av_opt_set(c->priv_data, "crf", "20", 0);
+	c->gop_size = 100;


Some of these should be configurable by the users as well, no?

There are many parameters in libavcodec - I included only a couple which I thought were the most important in this initial commit. I think other parameters should be made configurable - it is just a matter of finding out what the important ones are from mine/others' experience.

I feel like bit_rate, time_base and framerate at least should be 🤔

ericonr · 2024-03-06T14:13:53Z

supportApp/videoCompressionSrc/codec_tools.c

+	if (ret < 0) {
+		fprintf(stderr, "Could not open codec: %s\n", av_err2str(ret));
+		exit(1);
+	}


Failures in library code shouldn't cause the IOC to exit, right?

ericonr · 2024-03-06T14:16:08Z

supportApp/videoCompressionSrc/codec_tools.c

+		else if (ret < 0) {
+			fprintf(stderr, "Error during encoding\n");
+			exit(1);


An encoding error shouldn't cause the IOC to exit.

ericonr · 2024-03-06T14:21:59Z

supportApp/videoCompressionSrc/video_compression.c

+int H264_decompress(const char* source, char* dest, int originalSize)
+{
+
+        int width, height;
+        decompress_buffer(source, originalSize, dest, &width, &height);
+	return width*height;
+
+}


This function isn't declared in the header.

I see now that it's used in ADViewer. IMO it still should be in the header, as an exported interface.

ericonr · 2024-03-06T14:23:59Z

supportApp/videoCompressionSrc/video_compression.h

+extern "C" {
+#endif
+
+int H264_compress(const char* source, char* dest, int x_size, int y_size);


Shouldn't the size for dest be passed to this function?

I'd also feel more comfortable with the type in use being unsigned char rather than char. Pixels won't have negative values after all.

The only reason H264_compress would need the size of dest would be if it did the checking to see if the dest buffer is large enough. The H264_compress function is currently implemented under the assumption that the dest buffer is large enough to store the compressed image. Whether it is large enough depends on the upper bound function being accurate enough (which I admit this is unknown to me), since this is what ensures that dest is large enough. So far I have not noticed issues here, but I will do testing to further investigate this.

Edit: It seems there is checking done by the compress_buffer function it calls. dest buffer size is calculated in that function by multiplying image dimensions. Then there is a check to make sure the compressed size is small enough before copying. I will do testing later.

bsobhani · 2024-03-06T18:10:53Z

Erico, thank you for the feedback. I have responded to some of your comments. When I have time I will take a more thorough look at the pull request and probably make some additional commits.

…alled in ADCore without lock

…ome functions to take CodecContext object as a parameter

…context object as parameter instead of global context object. Also, make interfaces to these functions that take context as void pointer so that files that call these functions do not need to include libav headers

…sion, for compatibility. yuv422 seems to have issues for certain setups (openh264). change to yuv420 until further investigation made set_param_int function

This was referenced Dec 13, 2023

adding video compression (h264) areaDetector/ADCore#499

Open

adding video compression support (h264) areaDetector/ADViewers#25

Open

ericonr reviewed Mar 6, 2024

View reviewed changes

bsobhani force-pushed the video_compression branch from fc1b828 to 3547ad4 Compare April 1, 2024 06:00

bsobhani and others added 7 commits April 1, 2024 12:28

adding video compression (h264)

644369c

add lock/unlock for parameter writing, so that H264_compress can be c…

83809c3

…alled in ADCore without lock

create CodecContext struct that extends AVCodec context, and modify s…

d38274e

…ome functions to take CodecContext object as a parameter

make depreciated av_register function conditional upon libavcodec ver…

a54640d

…sion, for compatibility. yuv422 seems to have issues for certain setups (openh264). change to yuv420 until further investigation made set_param_int function

comment out global mutex and declare context initialization function

9415425

use correct context initialization function

81a8aa7

bsobhani force-pushed the video_compression branch from 3547ad4 to 81a8aa7 Compare April 1, 2024 16:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

adding video compression (h264) #44

adding video compression (h264) #44

bsobhani commented Dec 13, 2023

ericonr left a comment

ericonr Mar 6, 2024

ericonr Mar 6, 2024

ericonr Mar 6, 2024

bsobhani Mar 6, 2024

ericonr Mar 8, 2024

ericonr Mar 6, 2024

ericonr Mar 6, 2024

ericonr Mar 6, 2024

ericonr Mar 8, 2024

ericonr Mar 6, 2024

bsobhani Mar 6, 2024 •

edited

Loading

bsobhani commented Mar 6, 2024

adding video compression (h264) #44

Are you sure you want to change the base?

adding video compression (h264) #44

Conversation

bsobhani commented Dec 13, 2023

ericonr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bsobhani Mar 6, 2024 • edited Loading

Choose a reason for hiding this comment

bsobhani commented Mar 6, 2024

bsobhani Mar 6, 2024 •

edited

Loading