amazon-kvs-producer-sdk fails to create streams due to dependency commitish mismatches

[bavery22]

I am building amazon-kvs-producer-sdk-cpp_3.3.1.bb for Yocto kirkstone. I previously used 3.3.0 with dunfell. That worked fine in that I could create and push video streams from my device. With 3.3.1 I found that we were no longer able to create streams. We could only publish to streams that had already been created. We are building with gstreamer=ON.
There's a lot of numbers(commits) coming up, so I'll try to be clear.

the meta-aws I am using is 09db183 which is the kirkstone branch.
On that branch, the 3 recipes I need are using the following commits:
amazon-kvs-producer-pic_git.bb
SRCREV = "dc4bc6cb87cf97cd3b1dc3e8027e0c75cdfa13b3"
amazon-kvs-producer-sdk-c_1.4.1.bb
SRCREV = "80c74ac9200b58427a8fcb7782a03b1774020983"
amazon-kvs-producer-sdk-cpp_3.3.1.bb
SRCREV = "70f74f14cf27b09f71dc1889f36eb6e04cdd90a8"
.
Building/installing the above results in the gstreamer kvsssink not being able to create streams.
If we checkout
amazon-kvs-producer-sdk-cpp_3.3.1.bb
SRCREV = "70f74f14cf27b09f71dc1889f36eb6e04cdd90a8"
and look inside the CMake/Dependencies/libkvscproducer-CMakeLists.txt it calls for commit 79da5b6d6ad67560cc08530f967e9187a6d1c594 for the amazon-kvs-producer-sdk-c repo.
If we pull the amazon-kvs-producer-sdk-c repo and checkout the 79da5b commit and look inside the CMake/Dependencies/libkvspic-CMakeLists.txt we see that it calls for commit c8325887faa3a4a296c4367b281c778be69875b6 from the amazon-kvs-producer-pic repo.

If I use a bbappend to override the SRCREV for both the producer-sdk-c and the producer-pic to be consistent with the dependency chain in the CMake files, I can create and push video streams.

I am assuming that we should still be able to create streams from the device on kirkstone and that this is just a version mismatch and not a desired change in behavior?

I'm happy to submit the SRCREV overrides as a PR if that is desired/correct.

[thomas-roos]

Hi,
did check this with the service team responsible for the software and you're totally right.
Will change this asap as you suggested - will keep this issue open till it's fixed.
Thank you for this!

[thomas-roos]

could you please check if my commit fix your problem.
Thank you!

[bavery22]

So, the commit looks good. But I cannot build it to test. I tried building meta-aws/master-next with kirkstone and it failed to build with a linking error:
aarch64-poky-linux/11.3.0/ld: CMakeFiles/kvs_gstreamer_sample.dir/samples/kvs_gstreamer_sample.cpp.o: undefined reference to symbol 'strtoi32'
I am adding the following in my bbappend since we need gstreamer. It looks like the gstreamer sample is missing an include or a define for STRTOI32.

generated error calling bitbake amazon-kvs-producer-sdk-cpp

This is my amazon-kvs-producer-sdk-cpp_%.bbappend

DEPENDS:append = " curl log4cplus openssl gstreamer1.0 gstreamer1.0-plugins-base "
EXTRA_OECMAKE:remove = "-DBUILD_GSTREAMER_PLUGIN=OFF"
EXTRA_OECMAKE:append = "-DBUILD_GSTREAMER_PLUGIN=ON"


FILES:${PN}:append = " ${libdir} "
FILES:${PN}:append = " ${libdir}/gstreamer-1.0 "
FILES:${PN}:append = " ${libdir}/gstreamer-1.0/libgstkvssink.so "
FILES:${PN}:append = " ${libdir}/libKinesisVideoProducer.so "


do_install:append() {
    install -d ${D}${libdir}
    install -d ${D}${libdir}/gstreamer-1.0
    install -m 755 ${B}/libgstkvssink.so                                            ${D}${libdir}/gstreamer-1.0/libgstkvssink.so
    install -m 755 ${B}/libKinesisVideoProducer.so                                  ${D}${libdir}/libKinesisVideoProducer.so
}

`

[thomas-roos]

my recent commit now disable building the test and add an option the configure gstreamer via packageconfig. This should make your bbappend obsolete. Please try to confirm.

[bavery22]

Using kirkstone, I built and installed using t08d62031c3db8 on master-next.
All I needed was

amazon-kvs-producer-sdk-cpp_%.bbappend:
PACKAGECONFIG:append = " gstreamer "

ant the resulting binary ran, created streams on the fly, and the video was viewable.

ty!!!

[thomas-roos]

cool, this change will be merged into kirkstone till tmr.
Also be aware of setting PACKAGECONFIG in local.con etc.
https://docs.yoctoproject.org/ref-manual/variables.html?highlight=packageconfig#term-PACKAGECONFIG

[vivien]

Hi there, I'd like to reopen the issue, since I am getting the kvs_gstreamer_sample.cpp.o: undefined reference to symbol 'strtoi32' error with PACKAGECONFIG:append = " gstreamer" on the mickledore branch. Is there a pending patch that I must add to SRC_URI?

[thomas-roos]

Hi,
thanks for reaching out - probably a backport from scarthgap or master is necessary. Can you test if that works for you?
https://github.com/aws4embeddedlinux/meta-aws/tree/scarthgap/recipes-sdk/amazon-kvs-producer-sdk
Also consider switching to a newer Yocto release such as scarthgap, as mickledore release is EOL already and is not actively maintained.

[vivien]

Hi,

thanks for reaching out - probably a backport from scarthgap or master is necessary. Can you test if that works for you?

https://github.com/aws4embeddedlinux/meta-aws/tree/scarthgap/recipes-sdk/amazon-kvs-producer-sdk

Also consider switching to a newer Yocto release such as scarthgap, as mickledore release is EOL already and is not actively maintained.

Hi, thanks for your reply. Bumping yocto and supporting the latest releases is planned, unfortunately the versions to support on our side is determined by the various BSP layers that we depend on, some of them still provide support only for up to mickledore at the moment.

Can you point to me the back port fix you are referring to? If it is cherry-pickable, I can test it and suggest to apply it on the meta-aws mickledore branch, otherwise I'll bbappend the fix onto 3.4.1 on my side.

[thomas-roos]

Hi, meant just copying the whole folder: https://github.com/aws4embeddedlinux/meta-aws/tree/scarthgap/recipes-sdk/amazon-kvs-producer-sdk to mickledore and see if that works.

[vivien]

@thomas-roos what I've done is used the scarthgap branch for meta-aws and force LAYERSERIES_COMPAT_meta-aws:prepend = "mickledore ". The compilation now fails with undefined reference to symbol 'globalThreadSleep'.

[thomas-roos]

Give me 30min - I'm about to backport this.

[thomas-roos]

I do not have any compile issues - this is the backport PR: #9154
Once merged please test with mickledore-next branch and provide feedback if this solves your issue. Thank you

[thomas-roos]

Merged, and also merged to mickledore branch. Just do an update. Please provide feedback.

[vivien]

I've used your 2024-07-25_mickledore-next_backport-kvs branch and also latest mickledore and this results in the same undefined reference to symbol 'globalThreadSleep' error, as with the scarthgap branch.

It is interesting that you do not have an error though. Do you have a successful build after running bitbake -c cleansstate amazon-kvs-producer-sdk-cpp first? Also, do you have gstreamer appended in its PACKAGECONFIG in order to compile kvssink?

[thomas-roos]

I see now the issue. This is understood. Will dive into this.

[thomas-roos]

This is a Cmake link error as it seems: awslabs/amazon-kinesis-video-streams-producer-sdk-cpp#1197

Patched it here: #9168

please confirm

[vivien]

libgstkvssink.so is now being built, tested on both mickledore-next and scarthgap-next, feel free to add my Tested-by!