loadmodule Failed

[macarai]

I follow the requirements of Install redis-protobuf With Source Code.

system

• Ubuntu 19.10
• gcc version 9.2.1 20191008 (Ubuntu 9.2.1-9ubuntu2)
• protobuf-3.8.0-map-reflection
• Redis server v=5.0.5

redis.conf

################################## MODULES #####################################

# Load modules at startup. If the server is not able to load modules
# it will abort. It is possible to use multiple loadmodule directives.
#
# loadmodule /path/to/my_module.so
# loadmodule /path/to/other_module.so
loadmodule /usr/local/lib/rejson.so
loadmodule /usr/local/lib/libredis-protobuf.so --DIR /home/aicrobo/.redis/proto

redis logs

17878:M 05 Mar 2020 19:14:20.987 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
17878:M 05 Mar 2020 19:14:20.987 # Server initialized
17878:M 05 Mar 2020 19:14:20.987 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
17878:M 05 Mar 2020 19:14:20.987 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
17878:M 05 Mar 2020 19:14:20.987 # <ReJSON> JSON data type for Redis v1.0.4 [encver 0]
17878:M 05 Mar 2020 19:14:20.987 * Module 'ReJSON' loaded from /usr/local/lib/rejson.so
17878:M 05 Mar 2020 19:14:20.989 # Module /usr/local/lib/libredis-protobuf.so failed to load: /usr/local/lib/libredis-protobuf.so: undefined symbol: deflateInit2_
17878:M 05 Mar 2020 19:14:20.989 # Can't load module from /usr/local/lib/libredis-protobuf.so: server aborting

Help me, thank you!

[sewenew]

Hi @LHM69

failed to load: /usr/local/lib/libredis-protobuf.so: undefined symbol: deflateInit2_

I haven't met this problem before. It seems that your system doesn't have zlib installed. You can try installing zlib. Then restart Redis server to load redis-protobuf. If it still doesn't work, you can recompile protobuf and redis-protobuf after installing zlib.

redis-protobuf doesn't need zlib, however, protobuf needs zlib. If your system doesn't have zlib, it should fail to build. Are you running redis-protobuf on a different machine other than the one you build Redis-protobuf? If you do, you should install zlib on the machine where you run redis-protobuf.

If you still have any problem with it, feel free to let me know.

Regards

[macarai]

I did

CMakeLists.txt

find_library(PROTOBUF_LIB libprotobuf.a)
if (${CMAKE_SYSTEM_NAME} MATCHES "Darwin")
    target_link_libraries(${SHARED_LIB} -Wl,-force_load ${PROTOBUF_LIB})
    target_link_libraries(${SHARED_LIB} z)
else()
    target_link_libraries(${SHARED_LIB} -Wl,--whole-archive ${PROTOBUF_LIB} -Wl,--no-whole-archive)
endif()

to

find_library(PROTOBUF_LIB libprotobuf.a)
if (${CMAKE_SYSTEM_NAME} MATCHES "Darwin")
    target_link_libraries(${SHARED_LIB} -Wl,-force_load ${PROTOBUF_LIB})
    target_link_libraries(${SHARED_LIB} z)
else()
    target_link_libraries(${SHARED_LIB} -Wl,--whole-archive ${PROTOBUF_LIB} -Wl,--no-whole-archive)
    target_link_libraries(${SHARED_LIB} z) # here
endif()

redis logs

36813:M 06 Mar 2020 10:29:57.934 * Module 'ReJSON' loaded from /usr/local/lib/rejson.so
36813:M 06 Mar 2020 10:29:57.952 # <PB> failed to get file status: /home/aicrobo/.redis/proto
36813:M 06 Mar 2020 10:29:57.952 # Module /usr/local/lib/libredis-protobuf.so initialization failed. Module not loaded
36813:M 06 Mar 2020 10:29:57.952 # Can't load module from /usr/local/lib/libredis-protobuf.so: server aborting

redis.conf

/home/aicrobo/.redis/proto copy to /usr/lib/redis/proto

loadmodule /usr/local/lib/libredis-protobuf.so --DIR /home/aicrobo/.redis/proto

to

loadmodule /usr/local/lib/libredis-protobuf.so --dir /usr/lib/redis/proto

resis logs

38236:M 06 Mar 2020 10:53:29.299 # <ReJSON> JSON data type for Redis v1.0.4 [encver 0]
38236:M 06 Mar 2020 10:53:29.299 * Module 'ReJSON' loaded from /usr/local/lib/rejson.so
38236:M 06 Mar 2020 10:53:29.319 * Module 'PB' loaded from /usr/local/lib/libredis-protobuf.so
38236:M 06 Mar 2020 10:53:29.319 * DB loaded from disk: 0.000 seconds
38236:M 06 Mar 2020 10:53:29.319 * Ready to accept connections

Now works correctly, thanks

[sewenew]

Hi @LHM69

The first problem:

I cannot reproduce the problem with Ubuntu 18.04, and if I add target_link_libraries(${SHARED_LIB} z) to CMakeLists.txt it will failed to build. It's really strange.

I'll add a link to this issue, if others have the same problem they can try your solution. Thanks a lot!

The second problem

I think the reason is that the user running Redis do not have permission on /home/aicrobo/.redis/proto.

Thanks again for your work!

Regards

[JaiBalaj]

Hi @sewenew ,

I was trying to download and install redis-protobuf. I have no issue till cmake but make is not successful. Let me know if you have any input.

root@frcu32-1-5:/home/jthagaraja/redis-protobuf# ls -lrt
total 56
-rw-r--r-- 1 root root 23847 Aug 7 18:00 README.md
-rw-r--r-- 1 root root 11357 Aug 7 18:00 LICENSE
-rw-r--r-- 1 root root 164 Aug 7 18:00 Chinese.md
-rw-r--r-- 1 root root 1811 Aug 7 18:00 CMakeLists.txt
drwxr-xr-x 3 root root 4096 Aug 7 18:00 src
drwxr-xr-x 3 root root 4096 Aug 7 18:00 docker
drwxr-xr-x 3 root root 4096 Aug 7 18:00 test
root@frcu32-1-5:/home/jthagaraja/redis-protobuf#
root@frcu32-1-5:/home/jthagaraja/redis-protobuf#
root@frcu32-1-5:/home/jthagaraja/redis-protobuf# mkdir compile
root@frcu32-1-5:/home/jthagaraja/redis-protobuf# cd compile/
root@frcu32-1-5:/home/jthagaraja/redis-protobuf/compile# cmake ..
-- redis-protobuf version: 0.1.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- redis-protobuf build type: Release
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jthagaraja/redis-protobuf/compile
root@frcu32-1-5:/home/jthagaraja/redis-protobuf/compile#

root@frcu32-1-5:/home/jthagaraja/redis-protobuf/compile# make
[ 4%] Building CXX object CMakeFiles/shared.dir/src/sw/redis-protobuf/append_command.cpp.o
In file included from /home/jthagaraja/redis-protobuf/src/sw/redis-protobuf/append_command.h:24,
from /home/jthagaraja/redis-protobuf/src/sw/redis-protobuf/append_command.cpp:17:
/home/jthagaraja/redis-protobuf/src/sw/redis-protobuf/field_ref.h: In member function 'const google::protobuf::MapValueRef& sw::redis::pb::FieldRef::_get_map_value_const(Msg*, const google::protobuf::FieldDescriptor*, const google::protobuf::MapKey&) const':
/home/jthagaraja/redis-protobuf/src/sw/redis-protobuf/field_ref.h:357:42: error: invalid use of incomplete type 'const class google::protobuf::internal::GeneratedMessageReflection' [-Werror]
357 | const auto &map_base = reflection->GetRawgp::internal::MapFieldBase(*msg, field_desc);
| ^~
In file included from /usr/include/x86_64-linux-gnu/google/protobuf/map_entry_lite.h:43,
from /usr/include/x86_64-linux-gnu/google/protobuf/map_entry.h:35,
from /usr/include/x86_64-linux-gnu/google/protobuf/map_field.h:41,
from /home/jthagaraja/redis-protobuf/src/sw/redis-protobuf/field_ref.h:25:
/usr/include/x86_64-linux-gnu/google/protobuf/map.h:92:7: note: forward declaration of 'class google::protobuf::internal::GeneratedMessageReflection'
92 | class GeneratedMessageReflection;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/jthagaraja/redis-protobuf/src/sw/redis-protobuf/field_ref.h:357:44: error: expected 'template' keyword before dependent template name [-Werror=missing-template-keyword]
357 | const auto &map_base = reflection->GetRawgp::internal::MapFieldBase(*msg, field_desc);
| ^~~~~~
/home/jthagaraja/redis-protobuf/src/sw/redis-protobuf/field_ref.h:357:77: error: expected primary-expression before '>' token
357 | const auto &map_base = reflection->GetRawgp::internal::MapFieldBase(*msg, field_desc);

[JaiBalaj]

Is something wrong with the source code ? or am i missing any steps. Please let me know @sewenew

Thanks in advance.

[sewenew]

@JaiBalaj Looks like that you do not install the right version of protobuf. Since the official protobuf does not support map reflection, you need to install the one I modified. Check this for detail.

Regards

[JaiBalaj]

that error has been resolved but still i see not able to load the generated protobuf.so file.

3795241:M 09 Aug 2024 12:58:07.553 # Module /home/jthagaraja/redis-protobuf-0.1.0/compile/libredis-protobuf.so failed to load: /home/jthagaraja/redis-protobuf-0.1.0/compile/libredis-protobuf.so: undefined symbol: inflateInit2_
3795241:M 09 Aug 2024 12:58:07.553 # Can't load module from /home/jthagaraja/redis-protobuf-0.1.0/compile/libredis-protobuf.so: server aborting

[sewenew]

@JaiBalaj You can try to add target_link_libraries(${SHARED_LIB} z) to CMakeLists.txt file.

Check above comments in this issue for an example.

Regards

[JaiBalaj]

Hi @sewenew , Thanks issue has been resolved.