Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sonic-slave-stretch build failed #4315

Closed
tieguoevan opened this issue Mar 24, 2020 · 7 comments · Fixed by #4319
Closed

sonic-slave-stretch build failed #4315

tieguoevan opened this issue Mar 24, 2020 · 7 comments · Fixed by #4319
Assignees

Comments

@tieguoevan
Copy link

Description

sonic-slave-stretch cannot be built because debhelper from stretch-backports is conflicted with cmake from stretch

Step 10/42 : RUN apt-get -t stretch-backports install -y debhelper
 ---> Running in b4298563bd0d
Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  cmake-data libarchive13 libuv1
Use 'apt autoremove' to remove them.
The following additional packages will be installed:
  dh-autoreconf dpatch dwz quilt
The following packages will be REMOVED:
  cmake
The following NEW packages will be installed:
  dwz
The following packages will be upgraded:
  debhelper dh-autoreconf dpatch quilt
4 upgraded, 1 newly installed, 1 to remove and 147 not upgraded.
Need to get 1515 kB of archives.
After this operation, 15.9 MB disk space will be freed.

Without cmake, subsequent compilation will fail

Step 31/42 : RUN cd /usr/src/gtest && cmake . && make -C /usr/src/gtest
 ---> Running in fc9ae8ff35ea
�[91m/bin/sh: 1: cmake: not found
�[0mThe command '/bin/sh -c cd /usr/src/gtest && cmake . && make -C /usr/src/gtest' returned a non-zero code: 127
Sending build context to Docker daemon  28.67kB

sonic-jenkins can also see related logs
https://sonic-jenkins.westus2.cloudapp.azure.com/job/bldenv/job/sonic-slave-stretch/110/consoleFull

Steps to reproduce the issue:

  1. remove existed sonic-slave-stretch
  2. make configure
@qiluo-msft
Copy link
Collaborator

I think related to this bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=954852
The upstream is broken.

@lguohan
Copy link
Collaborator

lguohan commented Mar 25, 2020

I suspect not. it seems to be cause by debhelper

lgh@8bb9636d98fc:/sonic$ apt show debhelper
Package: debhelper
Version: 12.1.1~bpo9+1
Priority: optional
Section: devel
Maintainer: Debhelper Maintainers <debhelper@packages.debian.org>
Installed-Size: 1641 kB
Provides: debhelper-compat (= 9), debhelper-compat (= 10), debhelper-compat (= 11), debhelper-compat (= 12), dh-sequence-dwz, dh-sequence-installinitramfs, dh-sequence-systemd
Depends: autotools-dev, dh-autoreconf (>= 17~), dh-strip-nondeterminism (>= 0.028~), dpkg (>= 1.18.0~), dpkg-dev (>= 1.18.2~), dwz, file (>= 3.23), libdpkg-perl (>= 1.17.14), man-db, po-debconf, perl
Suggests: dh-make
Breaks: cmake (<< 3.9~), dh-systemd (<< 1.38), meson (<< 0.40.0~)
Replaces: dh-systemd (<< 1.38)
Download-Size: 1012 kB
APT-Manual-Installed: yes
APT-Sources: http://debian-archive.trafficmanager.net/debian stretch-backports/main amd64 Packages
Description: helper programs for debian/rules

N: There is 1 additional record. Please use the '-a' switch to see it

@lguohan
Copy link
Collaborator

lguohan commented Mar 25, 2020

this is a tricky one. here is my analysis:

  • smartmontools requires debhelper >= 11, therefore, we need to install debhelper 12.1.1 from stretch-backports.

  • debhelper in requires cmake (>= 3.9)

  • previously, we install cmake v13.3.2, which is ok. recently, cmake in debhelper is upgraded to v13.6.3 which requires libarchive13 (>= 3.3.3).

  • but libarchive13 is not available in stretch-backports.

Now, we cannot go back to cmake v13.3.2 since it is no longer available.

it looks like libarchive13 (3.4.0-2~bpo9+1) is in stretch-backports-sloppy

@lguohan lguohan linked a pull request Mar 25, 2020 that will close this issue
@gpaussabrcm
Copy link
Contributor

With this change, please keep a close watch on building src/libyang. When I tried this, the newer version of cmake seems to behave differently regarding SWIG, causing the following error:

make[3]: Entering directory '/sonic/src/libyang/libyang-1.0.73/build'                                                                                 
[ 96%] Building CXX object swig/python2/CMakeFiles/_yang2.dir/yangPYTHON_wrap.cxx.o                                                                   
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:3292:33: error: expected initializer before '.' token                        
 #  define SWIG_init    init_yang.so                                                                                                                  
                                 ^                                                                                                                    
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:102537:1: note: in expansion of macro 'SWIG_init'                            
 SWIG_init(void) {                                                                                                                                    
 ^                                                                                                                                                    
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:101971:24: warning: 'swig_const_table' defined but not used [-Wunused-variable]                                                                                                                                                    
 static swig_const_info swig_const_table[] = {                                                                                                        
                        ^~~~~~~~~~~~~~~~                                                                                                              
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:99421:20: warning: 'SwigMethods' defined but not used [-Wunused-variable]    
 static PyMethodDef SwigMethods[] = {                                                                                                                 
                    ^~~~~~~~~~~                                                                                                                       
swig/python2/CMakeFiles/_yang2.dir/build.make:62: recipe for target 'swig/python2/CMakeFiles/_yang2.dir/yangPYTHON_wrap.cxx.o' failed
make[3]: *** [swig/python2/CMakeFiles/_yang2.dir/yangPYTHON_wrap.cxx.o] Error 1
make[3]: Leaving directory '/sonic/src/libyang/libyang-1.0.73/build'
CMakeFiles/Makefile2:3866: recipe for target 'swig/python2/CMakeFiles/_yang2.dir/all' failed

Not sure if there are any other build errors in SONiC, but this is the first one I encountered.

For reference, here are the versions installed in my sonic-slave-stretch build container after making the Dockerfile.j2 changes:

# dpkg -l cmake libarchive13 debhelper  
Desired=Unknown/Install/Remove/Purge/Hold                           
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)                    
||/ Name                            Version              Architecture         Description
+++-===============================-====================-====================-====================================================================
ii  cmake                           3.16.3-1~bpo9+1      amd64                cross-platform, open-source make system                             
ii  debhelper                       12.1.1~bpo9+1        all                  helper programs for debian/rules                                    
ii  libarchive13:amd64              3.4.0-2~bpo9+1       amd64                Multi-format archive and compression library (shared library)       

-Greg

@lguohan
Copy link
Collaborator

lguohan commented Mar 25, 2020

ack. two solutions in my mind

  1. upgrade libyang to latest release 1.0.130 to see if there is issue.
  2. downgrade debhelper, and only install debhelper 12 before smartmontools and then download debhelper after build smartmontools.

@gpaussabrcm
Copy link
Contributor

Consider the following change as an additional patch in the src/libyang/patch series:

diff --git a/swig/CMakeLists.txt b/swig/CMakeLists.txt
index 0baa69ab..c0b75228 100644
--- a/swig/CMakeLists.txt
+++ b/swig/CMakeLists.txt
@@ -31,7 +31,9 @@ if(GEN_PYTHON_BINDINGS AND SWIG_FOUND)
             unset(PYTHON_INCLUDE_PATH CACHE)
             unset(PYTHON_EXT_SUFFIX CACHE)
             unset(PYTHON_MODULE_PATH CACHE)
-            set(PYTHON_EXT_SUFFIX ".so")
+            if(${CMAKE_VERSION} VERSION_LESS "3.16.0")
+                set(PYTHON_EXT_SUFFIX ".so")
+            endif()
         find_program(PYTHON_EXECUTABLE NAMES python${CUR_PYTHON_VERSION})
         execute_process(COMMAND ${PYTHON_EXECUTABLE} -c

The VERSION_LESS check most likely isn't exactly right, but this is meant as an alternative to upgrading the libyang version, and is in addition to the sonic-slave-stretch/Dockerfile.j2 changes already proposed as a fix. I've not tried this with anything other than my one build that was previously unable to create the stretch base image, so it'll need some vetting.

@lguohan
Copy link
Collaborator

lguohan commented Mar 27, 2020

@gpaussabrcm, thanks for the investigation. @qiluo-msft has found the pre-built binary for 3.13.2. we are fixing using this approach. It is probably better to keep cmake the same version.

i think this time lesson learned. I did not realized so many project relies on the cmake. Did not realize cmake is such a basic tool. it is probably not a good idea to cmake from stretch-backports in the first place.

We should probably be more careful on accepting the pr that upgrades debhelper in the first place. after all, only one project requires debhelper >= 11.

lguohan added a commit that referenced this issue Mar 27, 2020
fix #4315

download and install cmake 3.13.2 from azure storage 

latest cmake 3.16.3 from stretch-backports breaks the libyang 1.0.73 build

Signed-off-by: Guohan Lu <lguohan@gmail.com>
rlhui pushed a commit that referenced this issue Mar 30, 2020
fix #4315

download and install cmake 3.13.2 from azure storage 

latest cmake 3.16.3 from stretch-backports breaks the libyang 1.0.73 build

Signed-off-by: Guohan Lu <lguohan@gmail.com>
taocy001 pushed a commit to CentecNetworks/sonic-buildimage that referenced this issue Apr 7, 2020
fix sonic-net#4315

download and install cmake 3.13.2 from azure storage

latest cmake 3.16.3 from stretch-backports breaks the libyang 1.0.73 build

Signed-off-by: Guohan Lu <lguohan@gmail.com>
tiantianlv pushed a commit to SONIC-DEV/sonic-buildimage that referenced this issue Apr 24, 2020
fix sonic-net#4315

download and install cmake 3.13.2 from azure storage 

latest cmake 3.16.3 from stretch-backports breaks the libyang 1.0.73 build

Signed-off-by: Guohan Lu <lguohan@gmail.com>
YuriiHordynskyi pushed a commit to YuriiHordynskyi/sonic-buildimage that referenced this issue May 18, 2020
fix sonic-net#4315

download and install cmake 3.13.2 from azure storage 

latest cmake 3.16.3 from stretch-backports breaks the libyang 1.0.73 build

Signed-off-by: Guohan Lu <lguohan@gmail.com>
vitalivanov pushed a commit to InterfaceMasters/sonic-buildimage that referenced this issue May 20, 2020
fix sonic-net#4315

download and install cmake 3.13.2 from azure storage 

latest cmake 3.16.3 from stretch-backports breaks the libyang 1.0.73 build

Signed-off-by: Guohan Lu <lguohan@gmail.com>
zhenggen-xu pushed a commit to zhenggen-xu/sonic-buildimage that referenced this issue May 22, 2020
fix sonic-net#4315

download and install cmake 3.13.2 from azure storage 

latest cmake 3.16.3 from stretch-backports breaks the libyang 1.0.73 build

Signed-off-by: Guohan Lu <lguohan@gmail.com>
xumia added a commit that referenced this issue Dec 28, 2022
Why I did it
The libyang cannot be built with the latest version of cmake.
Example: https://dev.azure.com/mssonic/build/_build/results?buildId=181862&view=logs&j=993d6e22-aeec-5c03-fa19-35ecba587dd9&t=d0538dec-1681-5ff8-bd45-c0de13be9706

[ 96%] Building CXX object swig/python2/CMakeFiles/_yang2.dir/yangPYTHON_wrap.cxx.o
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:3292:33: error: expected initializer before '.' token
 #  define SWIG_init    init_yang.so
                                 ^
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:102537:1: note: in expansion of macro 'SWIG_init'
 SWIG_init(void) {
 ^~~~~~~~~
/sonic/src/libyang/libyang-1.0.73/build/swig/python2/yangPYTHON_wrap.cxx:101971:24: warning: 'swig_const_table' defined but not used [-Wunused-variable]
 static swig_const_info swig_const_table[] = {

See relative issue: #4315

How I did it
Not upgrade the cmake, change to use the dbus-python 1.2.18 which is compatible with cmake 3.13.4

How to verify it
See https://dev.azure.com/mssonic/build/_build/results?buildId=194756&view=results
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants