Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Error on building heads: heads/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: multiple definition of `head'; x230-flash/util/sconfig/sconfig.tab.o #853

Closed
explit7 opened this issue Oct 16, 2020 · 25 comments

Comments

@explit7
Copy link

explit7 commented Oct 16, 2020

Hello colegues.
Have some problems to build heads on Debian 10
Maybe this is known?
Thanks

The sections containing CBFSes are: COREBOOT
    CC         bootblock/lib/cbfs.o
    CC         bootblock/lib/cbmem_console.o
    CC         bootblock/lib/delay.o
    CC         bootblock/lib/fmap.o
    CC         bootblock/lib/gcc.o
    CC         bootblock/lib/halt.o
    CC         bootblock/lib/hexdump.o
    CC         bootblock/lib/libgcc.o
    CC         bootblock/lib/memchr.o
    CC         bootblock/lib/memcmp.o
    CC         bootblock/lib/prog_loaders.o
    CC         bootblock/lib/prog_ops.o
    CC         bootblock/lib/reset.o
    CC         bootblock/lib/version.o
    HOSTCC     util/sconfig/sconfig (link)
/usr/bin/ld: x230-flash/util/sconfig/main.o:/media/sdb1/Git-Projects/heads/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: multiple definition of `head'; x230-flash/util/sconfig/sconfig.tab.o:/media/sdb1/Git-Projects/heads/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [util/sconfig/Makefile.inc:43: x230-flash/util/sconfig/sconfig] Error 1
make[2]: Leaving directory '/media/sdb1/Git-Projects/heads/build/coreboot-4.8.1'
make[1]: *** [Makefile:438: /media/sdb1/Git-Projects/heads/build/coreboot-4.8.1/x230-flash/.build] Error 1
make[1]: Leaving directory '/media/sdb1/Git-Projects/heads'
make: *** [Makefile:643: all] Error 2

@Thrilleratplay
Copy link
Contributor

I know I have seen that error before but can't remember the exact reason. You may be missing a dependency, try this and see if anything needs to be installed. If so, install it and try building again.

    apt-get install \
      bc \
      bison \
      build-essential \
      cpio \
      flex \
      libelf-dev \
      libncurses5-dev \
      pkg-config \
      texinfo \
      zlib1g-dev

@tlaurion
Copy link
Collaborator

tlaurion commented Oct 16, 2020

Quick reminder that for a Heads commit ID, dependencies for Debian 10 are currently covered under CircleCI under https://github.com/osresearch/heads/blob/master/.circleci/config.yml#L8-L11 as referred under heads-wiki https://github.com/osresearch/heads-wiki/blob/master/Building.md

If for some reason, there is corrupted artifacts from past builds (some patches were added for the same module dowloaded version after decompression of module archive), patches are only applied once per module version change.

The documentation might need to be clearer on how to create a cleaner build:
make BOARD=x230-flash real.clean

While you may want to add:
rm -rf ./build/*
and redo
make BOARD=x230-flash

And adding this to the precedent commands will rebuild also musl-cross-make:
rm -rf ./crossgcc
make BOARD=x230-flash

@explit7
Copy link
Author

explit7 commented Oct 16, 2020

I know I have seen that error before but can't remember the exact reason. You may be missing a dependency, try this and see if anything needs to be installed. If so, install it and try building again.

    apt-get install \
      bc \
      bison \
      build-essential \
      cpio \
      flex \
      libelf-dev \
      libncurses5-dev \
      pkg-config \
      texinfo \
      zlib1g-dev

Thanks for the quick answer!
All dependencies are installed...

bc is already the newest version (1.07.1-2+b2).
bison is already the newest version (2:3.7.2+dfsg-1).
build-essential is already the newest version (12.8).
cpio is already the newest version (2.13+dfsg-4).
libelf-dev is already the newest version (0.181-1).
flex is already the newest version (2.6.4-8).
libncurses5-dev is already the newest version (6.2+20200918-1).
pkg-config is already the newest version (0.29.2-1).
texinfo is already the newest version (6.7.0.dfsg.2-5).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-2).

Will try to do

 make BOARD=x230-flash real.clean
 rm -rf ./build/*
 rm -rf ./crossgcc
 make BOARD=x230-flash


@Thrilleratplay
Copy link
Contributor

@explit7 Before you do, try installing all of the packages @tlaurion mentioned in the circleci/config.yml. cmake and autoconf may be needed. Try this before removing everything in build and crossgcc, it will save a lot of time.

@explit7
Copy link
Author

explit7 commented Oct 16, 2020

@explit7 Before you do, try installing all of the packages @tlaurion mentioned in the circleci/config.yml. cmake and autoconf may be needed. Try this before removing everything in build and crossgcc, it will save a lot of time.

Thanks, already installed, but in debian 10 there are some problems with the package python. as i understand, package python is deprecated, so i installed python2 instead and made symlink to python.

sudo apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'acpica-tools' instead of 'iasl'
Note, selecting 'libncurses-dev' instead of 'ncurses-dev'
acpica-tools is already the newest version (20200528-1).
autoconf is already the newest version (2.69-11.1).
bc is already the newest version (1.07.1-2+b2).
bison is already the newest version (2:3.7.2+dfsg-1).
build-essential is already the newest version (12.8).
bzip2 is already the newest version (1.0.8-4).
ccache is already the newest version (3.7.12-1).
cmake is already the newest version (3.18.2-1).
cpio is already the newest version (2.13+dfsg-4).
libelf-dev is already the newest version (0.181-1).
flex is already the newest version (2.6.4-8).
git is already the newest version (1:2.28.0-1).
gnat is already the newest version (9).
gnupg is already the newest version (2.2.20-1).
libusb-1.0-0-dev is already the newest version (2:1.0.23-2).
m4 is already the newest version (1.4.18-4).
nasm is already the newest version (2.15.05-1).
libncurses-dev is already the newest version (6.2+20200918-1).
patch is already the newest version (2.7.6-6).
patch set to manually installed.
pkg-config is already the newest version (0.29.2-1).
texinfo is already the newest version (6.7.0.dfsg.2-5).
uuid-dev is already the newest version (2.36-3+b1).
wget is already the newest version (1.20.3-1+b3).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-2).
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 python : PreDepends: python-minimal (= 2.7.17-2) but it is not going to be installed
          Depends: libpython-stdlib (= 2.7.17-2) but it is not going to be installed
          Depends: python2 (= 2.7.17-2) but 2.7.18-2 is to be installed
E: Unable to correct problems, you have held broken packages.


@explit7
Copy link
Author

explit7 commented Oct 18, 2020

unfortunately the same result after

make BOARD=x230-flash real.clean
 rm -rf ./build/*
 rm -rf ./crossgcc
 make BOARD=x230-flash

@tlaurion
Copy link
Collaborator

@explit7 this is weird, since CircleCI builds are based on debian-10. Will check again on local QubesOS debian-10 based AppVM on clean build later on

@tlaurion
Copy link
Collaborator

tlaurion commented Nov 13, 2020

@explit7 @Thrilleratplay
Will do with build trace

user@x230-master:~/heads$ git fetch osresearch 
remote: Enumerating objects: 13, done.
remote: Counting objects: 100% (13/13), done.
remote: Compressing objects: 100% (7/7), done.
Unpacking objects: 100% (18/18), done.
remote: Total 18 (delta 7), reused 10 (delta 6), pack-reused 5
From https://github.com/osresearch/heads
   89c4577..5614525  master     -> osresearch/master
user@x230-master:~/heads$ git checkout osresearch/master
Previous HEAD position was 89c4577 coreboot configs: remove CONFIG_LOCALVERSION artifacts in conformity of  https://github.com/osresearch/heads/commit/bd7a945bbbf586ceb7c83785ef36d95f0adf7a15
HEAD is now at 5614525 boards/librem*: enable automatic booting
user@x230-master:~/heads$ git reset --hard
HEAD is now at 5614525 boards/librem*: enable automatic booting
user@x230-master:~/heads$ grep apt .circleci/config.yml
            apt update
            apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync
user@x230-master:~/heads$ sudo apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'acpica-tools' instead of 'iasl'
Note, selecting 'libncurses-dev' instead of 'ncurses-dev'
acpica-tools is already the newest version (20181213-1).
autoconf is already the newest version (2.69-11).
automake is already the newest version (1:1.16.1-4).
bc is already the newest version (1.07.1-2+b1).
bison is already the newest version (2:3.3.2.dfsg-1).
build-essential is already the newest version (12.6).
bzip2 is already the newest version (1.0.6-9.2~deb10u1).
ccache is already the newest version (3.6-1).
cmake is already the newest version (3.13.4-1).
cpio is already the newest version (2.12+dfsg-9).
doxygen is already the newest version (1.8.13-10).
libelf-dev is already the newest version (0.176-1.1).
flex is already the newest version (2.6.4-6.2).
git is already the newest version (1:2.20.1-2+deb10u3).
gnat is already the newest version (8.0.1).
gnupg is already the newest version (2.2.12-1+deb10u1).
graphviz is already the newest version (2.40.1-6).
libdigest-sha-perl is already the newest version (6.02-1+b1).
libtool is already the newest version (2.4.6-9).
libusb-1.0-0-dev is already the newest version (2:1.0.22-2).
m4 is already the newest version (1.4.18-2).
nasm is already the newest version (2.14-1).
libncurses-dev is already the newest version (6.1+20181013-2+deb10u2).
patch is already the newest version (2.7.6-3+deb10u1).
pkg-config is already the newest version (0.29-6).
python is already the newest version (2.7.16-1).
rsync is already the newest version (3.1.3-6).
libudev-dev is already the newest version (241-7~deb10u4).
libudev1 is already the newest version (241-7~deb10u4).
udev is already the newest version (241-7~deb10u4).
texinfo is already the newest version (6.5.0.dfsg.1-4+b1).
uuid-dev is already the newest version (2.33.1-0.1).
wget is already the newest version (1.20.1-1.1).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-1).
The following packages were automatically installed and are no longer required:
  gconf2 libqt5sensors5 libqt5webkit5 lightning linux-headers-4.19.0-10-amd64 linux-headers-4.19.0-10-common linux-headers-4.19.0-5-amd64
  linux-headers-4.19.0-5-common linux-headers-4.19.0-6-amd64 linux-headers-4.19.0-6-common linux-headers-4.19.0-8-amd64
  linux-headers-4.19.0-8-common linux-headers-4.19.0-9-amd64 linux-headers-4.19.0-9-common linux-image-4.19.0-10-amd64 linux-image-4.19.0-5-amd64
  linux-image-4.19.0-6-amd64 linux-image-4.19.0-8-amd64 linux-image-4.19.0-9-amd64 mpv phantomjs python-dbus python-qubesdb python-xdg
  python3-pyxattr riot-desktop rtmpdump
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
user@x230-master:~/heads$ make BOARD=x230 CPUS=2 real.clean
2020-11-13 13:05:19-05:00 Wrong gawk detected: 4.2.1
Makefile:202: warning: overriding recipe for target 'all'
Makefile:68: warning: ignoring old recipe for target 'all'
for dir in \
	x230 cryptsetup-1.7.3 busybox-1.28.0 coreboot-4.8.1/x230 coreboot-4.8.1/3rdparty/blobs libksba-1.4.0 kexec-tools-2.0.20 pinentry-1.1.0 lvm2.2.02.168 popt-1.16 hotp-verification-c0956cfa085bcfc2500c1085dad350a440ccbe40/hidapi cairo-1.14.12 mbedtls-2.4.2 npth-1.6 libgcrypt-1.8.6 qrencode-3.4.4 pixman-0.34.0 fbwhiptail-0f14a409735b71c219e0b9b3ee63cdae709ba143 pciutils-3.5.4 libassuan-2.5.3 libusb-1.0.21 linux-4.14.62/linux-x230 gnupg-2.2.21 flashtools-40d5170e84a3822552df7a500cd00aa870fdfe76 libpng-1.6.34 tpmtotp-18b860fdcf5a55537c8395b891f2b2a5c24fc00a libusb-compat-0.1.5 libgpg-error-1.37 flashrom-b1f858f65b2abd276542650d8cb9e382da258967 util-linux-2.29.2 zlib-1.2.11 \
	 \
; do \
	if [ ! -z "$dir" ]; then \
		rm -rf "build/$dir"; \
	fi; \
done
cd install && rm -rf -- *
user@x230-master:~/heads$ rm -rf crossgcc/ build/*
user@x230-master:~/heads$ tar zxvf archive.tar.gz ./
./packages/libgcrypt-1.8.3.tar.bz2
./packages/libassuan-2.5.1.tar.bz2
./packages/libassuan-2.5.3.tar.bz2
./packages/cryptsetup-1.7.5.tar.xz
./packages/cryptsetup-1.7.3.tar.xz
./packages/musl-1.1.15.tar.gz
./packages/linux-5.4.30.tar.xz
./packages/cairo-1.14.12.tar.xz
./packages/LVM2.2.02.168.tgz
./packages/coreboot-4.12.tar.xz
./packages/gnupg-2.2.20.tar.bz2
./packages/dropbear-2016.74.tar.bz2
./packages/busybox-1.28.0.tar.bz2
./packages/libpng-1.6.34.tar.gz
./packages/libusb-1.0.21.tar.bz2
./packages/gnupg-2.2.21.tar.bz2
./packages/linux-4.19.114.tar.xz
./packages/popt-1.16.tar.gz
./packages/libksba-1.3.5.tar.bz2
./packages/purism-blobs-f53d4074a81c70352d39839884caac20181274d1.tar.gz
./packages/linux-b87b58e1b057a2706d422fbdc76aa34309c6c90b.tar.xz
./packages/coreboot-4.8.1.tar.xz
./packages/mbedtls-2.4.2-gpl.tgz
./packages/libgcrypt-1.8.4.tar.bz2
./packages/coreboot-blobs-4.12.tar.xz
./packages/linux-4.19.63.tar.xz
./packages/libgpg-error-1.39.tar.bz2
./packages/libksba-1.4.0.tar.bz2
./packages/libgcrypt-1.8.6.tar.bz2
./packages/libgcrypt-1.8.5.tar.bz2
./packages/slang-2.3.1a.tar.bz2
./packages/zlib-1.2.11.tar.gz
./packages/libgpg-error-1.32.tar.bz2
./packages/libgpg-error-1.37.tar.bz2
./packages/pixman-0.34.0.tar.gz
./packages/linux-4.14.62.tar.xz
./packages/gnupg-2.2.10.tar.bz2
./packages/coreboot-blobs-4.8.1.tar.xz
./packages/util-linux-2.29.2.tar.xz
./packages/cryptsetup-2.3.3.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/acpica-unix2-20200110.tar.gz
./build/coreboot-4.12/util/crossgcc/tarballs/gmp-6.1.2.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/binutils-2.33.1.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/gcc-8.3.0.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/mpfr-4.0.2.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/mpc-1.1.0.tar.gz
./build/coreboot-4.8.1/util/crossgcc/tarballs/acpica-unix2-20180531.tar.gz
user@x230-master:~/heads$ 
user@x230-master:~/heads$ 
user@x230-master:~/heads$ make BOARD=x230 CPUS=2 
2020-11-13 13:07:06-05:00 Wrong gawk detected: 4.2.1
Makefile:202: warning: overriding recipe for target 'all'
Makefile:68: warning: ignoring old recipe for target 'all'
tar xf "/home/user/heads/packages/gawk-4.2.1.tar.xz" -C "/home/user/heads/build"
touch "/home/user/heads/build/gawk-4.2.1/.extract"
touch "/home/user/heads/build/gawk-4.2.1/.patch"
cd "/home/user/heads/build/gawk-4.2.1/" ; \
./configure 2>&1 \
| tee "/home/user/heads/build/log/gawk.configure.log" \
> /dev/null
touch "/home/user/heads/build/gawk-4.2.1/.configured"
make -C "/home/user/heads/build/gawk-4.2.1/"  \
	2>&1 \
	| tee "/home/user/heads/build/log/gawk.log" \
	> /dev/null

[...]

@tlaurion
Copy link
Collaborator

tlaurion commented Nov 13, 2020

@Thrilleratplay For cache building clarifications on CircleCI:

Then

@Thrilleratplay So unless musl-cross-make git commit changes happened in its module (that module text has changed), that cache is reused. In both caches.

I have resetted the cache and also rebuilding from CI, while changing the environment variable in CI which is used here and there which is inserted in build through the Environement Variables section of CI for future builds, basically pushing a new .Environment.CACHE_VERSION that will be picked up on next build.

But here, at each builds the docker debian:10 container is used, build dependencies and updates are installed, git is cloned and positioned at latest commit, measurements are taken on modules, and the largest cache matching actual modules+patches measurements is reused if key matches on modules measurements.

If i'm doing something wrong here please let me know, otherwise, we are not recreating a docker image here, we are trying to simulate a debian-10 compilation but not for a signel board, but for all boards, reusing the caches to save time.

@tlaurion
Copy link
Collaborator

tlaurion commented Nov 13, 2020

@Thrilleratplay @explit7 clean build following precedent information is happening here

You can see:

@explit7
Copy link
Author

explit7 commented Nov 13, 2020

thanks. Will also try to fire up ubuntu vm to build it.... if nothing helps

@tlaurion
Copy link
Collaborator

tlaurion commented Nov 13, 2020

My question is more: what are the differences between docker 10, QubesOS debian-10 Template and native debian-10 installation to cause the issue.

As specified under #571, I think its time to move out of trying to build on all OSes of the world, and go docker, where that docker should probably be guix-buildstack as @daym is currently working on, so that end user instructions are clear, implementable in CIs, limiting impacts of host differences and permitting to replicate the results of today's commit in 5 years from now, pulling the same buildstack revision and permitting to fixate toolstack revisions desired for different coreboot versions, for example.

@Thrilleratplay
Copy link
Contributor

Thrilleratplay commented Nov 13, 2020

@tlaurion You are not doing anything wrong, the caching is misleading. apt-update updates the repository information. I suspect, and correct me if I am wrong, CircleCI will not pull the original Debian:10 container again and rerun apt-update and apt-install dependencies but instead pull from its cache the steps taken under Install Dependencies as this has not been changed in the configuration script.

This means the package versions used in CI are not necessarily the same as those users try to build Heads locally. A package may be updated twice in a month where once version creates an error and a second one does not. The result being intermittent bugs in the build process. A preconfigured docker image with all of the necessary packages installed hosted on a registry, similar to the coreboot-sdk image, so that CI and users could use the same base would solve these issues. Depending on the direction #571 it would likely resolve itself.

If this explanation is not clear, please let me know.

@tlaurion
Copy link
Collaborator

tlaurion commented Nov 13, 2020

@tlaurion You are not doing anything wrong, the caching is misleading. apt-update updates the repository information. I suspect, and correct me if I am wrong, CircleCI will not pull the original Debian:10 container again and rerun apt-update and apt-install dependencies but instead pull from its cache the steps taken under Install Dependencies as this has not been changed in the configuration script.

@Thrilleratplay : No! It actually pulls the docker image, calls apt update and then install dependencies at each build. (This is not cached!!!) Please click the links here so that we are all at the same page.

This means the package versions used in CI are not necessarily the same as those users try to build Heads locally.

Well. Users should also update their packages and install the dependencies the same way the CI does at each build. I mean. OS is updated, right? On debian-10, this is the only way we have to detect Host releated problems, and why we want to get away of those dependencies.

A package may be updated twice in a month where once version creates an error and a second one does not. The result being intermittent bugs in the build process. A preconfigured docker image with all of the necessary packages installed hosted on a registry, similar to the coreboot-sdk image, so that CI and users could use the same base would solve these issues. Depending on the direction #571 it would likely resolve itself.

I do not find the references here but coreboot-sdk is not reproducible right now. (searching source and discussions)

If this explanation is not clear, please let me know.

@Thrilleratplay
Copy link
Contributor

@tlaurion I looked back through other CircleCI builds and it is updating and installing the packages each build. My apologies. I assumed it would cache this if subsequent layers were also pulling from cache.

Yes, users should be able to update their OS but if the goal is reproducible build, changes in package versions would likely result in change to the checksum of the compiled rom.

It is not that coreboot-sdk would be reproducible if rebuilt, it is the fact that the same build is shared. Anyone can do a docker pull coreboot/coreboot-sdk:6065f616eb and have the exact same build environment. This would be the same environment today or 5 years from now. The same way docker pull coreboot/coreboot-sdk:1.35 creates the same build environment it did 5 years ago.

@tlaurion
Copy link
Collaborator

tlaurion commented Nov 13, 2020

@tlaurion I looked back through other CircleCI builds and it is updating and installing the packages each build. My apologies. I assumed it would cache this if subsequent layers were also pulling from cache.

@Thrilleratplay : Glad we finally have a trace of the actual problem we are trying to resolve. :)

Yes, users should be able to update their OS but if the goal is reproducible build, changes in package versions would likely result in change to the checksum of the compiled rom.

And here lies the problem. Theoretically, we would expect host buildstack versions to be sticked to major versions where impact should be limited. Problems lies in having to support really old versions of coreboot now (4.8.1 because #709 and coreboot 4.11 #867) where problems will definitely arise, unless we can switch buildstack prior of building those board in CI or locally per proper instructions against a docker image permitting that, with CI instructions and doc explaining and applying the process.

It is not that coreboot-sdk would be reproducible if rebuilt, it is the fact that the same build is shared. Anyone can do a docker pull coreboot/coreboot-sdk:6065f616eb and have the exact same build environment. This would be the same environment today or 5 years from now. The same way docker pull coreboot/coreboot-sdk:1.35 creates the same build environment it did 5 years ago.

So maybe solution 1 here is the way to go. While that docker being too recent won't build coreboot 4.8.1 and too old won't build 4.12+. Here lies the problem and hopes into solution 2.

@Thrilleratplay
Copy link
Contributor

@tlaurion Solution #1 or Solution #2 would solve this. As long as an image of the exact same build environment can be used by both the CI and end users, this should provide the ability to create reproducible builds.

@tlaurion
Copy link
Collaborator

tlaurion commented Nov 15, 2020

@Thrilleratplay @explit7 clean build following precedent information is happening here

You can see:

* [Docker image downloaded here from docker hub](https://app.circleci.com/pipelines/github/tlaurion/heads/607/workflows/b57ddc4b-e161-4517-81e2-9327dcafcfb1/jobs/655/parallel-runs/0/steps/0-0)

* [No cache was downloaded nor applied since cache key signature didn't match](https://app.circleci.com/pipelines/github/tlaurion/heads/607/workflows/b57ddc4b-e161-4517-81e2-9327dcafcfb1/jobs/655) since `.Environment.CACHE_VERSION` was changed in CI so no cache signature literals matched.

* Build is [happening](https://app.circleci.com/pipelines/github/tlaurion/heads/607/workflows/b57ddc4b-e161-4517-81e2-9327dcafcfb1/jobs/655) but is expected to take around 3h from now.

@Thrilleratplay @explit7 :

@tlaurion
Copy link
Collaborator

@explit7 @Thrilleratplay
Will do with build trace

user@x230-master:~/heads$ git fetch osresearch 
remote: Enumerating objects: 13, done.
remote: Counting objects: 100% (13/13), done.
remote: Compressing objects: 100% (7/7), done.
Unpacking objects: 100% (18/18), done.
remote: Total 18 (delta 7), reused 10 (delta 6), pack-reused 5
From https://github.com/osresearch/heads
   89c4577..5614525  master     -> osresearch/master
user@x230-master:~/heads$ git checkout osresearch/master
Previous HEAD position was 89c4577 coreboot configs: remove CONFIG_LOCALVERSION artifacts in conformity of  https://github.com/osresearch/heads/commit/bd7a945bbbf586ceb7c83785ef36d95f0adf7a15
HEAD is now at 5614525 boards/librem*: enable automatic booting
user@x230-master:~/heads$ git reset --hard
HEAD is now at 5614525 boards/librem*: enable automatic booting
user@x230-master:~/heads$ grep apt .circleci/config.yml
            apt update
            apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync
user@x230-master:~/heads$ sudo apt install -y build-essential zlib1g-dev uuid-dev libdigest-sha-perl libelf-dev bc bzip2 bison flex git gnupg iasl m4 nasm patch python wget gnat cpio ccache pkg-config cmake libusb-1.0-0-dev autoconf texinfo ncurses-dev doxygen graphviz udev libudev1 libudev-dev automake libtool rsync
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'acpica-tools' instead of 'iasl'
Note, selecting 'libncurses-dev' instead of 'ncurses-dev'
acpica-tools is already the newest version (20181213-1).
autoconf is already the newest version (2.69-11).
automake is already the newest version (1:1.16.1-4).
bc is already the newest version (1.07.1-2+b1).
bison is already the newest version (2:3.3.2.dfsg-1).
build-essential is already the newest version (12.6).
bzip2 is already the newest version (1.0.6-9.2~deb10u1).
ccache is already the newest version (3.6-1).
cmake is already the newest version (3.13.4-1).
cpio is already the newest version (2.12+dfsg-9).
doxygen is already the newest version (1.8.13-10).
libelf-dev is already the newest version (0.176-1.1).
flex is already the newest version (2.6.4-6.2).
git is already the newest version (1:2.20.1-2+deb10u3).
gnat is already the newest version (8.0.1).
gnupg is already the newest version (2.2.12-1+deb10u1).
graphviz is already the newest version (2.40.1-6).
libdigest-sha-perl is already the newest version (6.02-1+b1).
libtool is already the newest version (2.4.6-9).
libusb-1.0-0-dev is already the newest version (2:1.0.22-2).
m4 is already the newest version (1.4.18-2).
nasm is already the newest version (2.14-1).
libncurses-dev is already the newest version (6.1+20181013-2+deb10u2).
patch is already the newest version (2.7.6-3+deb10u1).
pkg-config is already the newest version (0.29-6).
python is already the newest version (2.7.16-1).
rsync is already the newest version (3.1.3-6).
libudev-dev is already the newest version (241-7~deb10u4).
libudev1 is already the newest version (241-7~deb10u4).
udev is already the newest version (241-7~deb10u4).
texinfo is already the newest version (6.5.0.dfsg.1-4+b1).
uuid-dev is already the newest version (2.33.1-0.1).
wget is already the newest version (1.20.1-1.1).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-1).
The following packages were automatically installed and are no longer required:
  gconf2 libqt5sensors5 libqt5webkit5 lightning linux-headers-4.19.0-10-amd64 linux-headers-4.19.0-10-common linux-headers-4.19.0-5-amd64
  linux-headers-4.19.0-5-common linux-headers-4.19.0-6-amd64 linux-headers-4.19.0-6-common linux-headers-4.19.0-8-amd64
  linux-headers-4.19.0-8-common linux-headers-4.19.0-9-amd64 linux-headers-4.19.0-9-common linux-image-4.19.0-10-amd64 linux-image-4.19.0-5-amd64
  linux-image-4.19.0-6-amd64 linux-image-4.19.0-8-amd64 linux-image-4.19.0-9-amd64 mpv phantomjs python-dbus python-qubesdb python-xdg
  python3-pyxattr riot-desktop rtmpdump
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 8 not upgraded.
user@x230-master:~/heads$ make BOARD=x230 CPUS=2 real.clean
2020-11-13 13:05:19-05:00 Wrong gawk detected: 4.2.1
Makefile:202: warning: overriding recipe for target 'all'
Makefile:68: warning: ignoring old recipe for target 'all'
for dir in \
	x230 cryptsetup-1.7.3 busybox-1.28.0 coreboot-4.8.1/x230 coreboot-4.8.1/3rdparty/blobs libksba-1.4.0 kexec-tools-2.0.20 pinentry-1.1.0 lvm2.2.02.168 popt-1.16 hotp-verification-c0956cfa085bcfc2500c1085dad350a440ccbe40/hidapi cairo-1.14.12 mbedtls-2.4.2 npth-1.6 libgcrypt-1.8.6 qrencode-3.4.4 pixman-0.34.0 fbwhiptail-0f14a409735b71c219e0b9b3ee63cdae709ba143 pciutils-3.5.4 libassuan-2.5.3 libusb-1.0.21 linux-4.14.62/linux-x230 gnupg-2.2.21 flashtools-40d5170e84a3822552df7a500cd00aa870fdfe76 libpng-1.6.34 tpmtotp-18b860fdcf5a55537c8395b891f2b2a5c24fc00a libusb-compat-0.1.5 libgpg-error-1.37 flashrom-b1f858f65b2abd276542650d8cb9e382da258967 util-linux-2.29.2 zlib-1.2.11 \
	 \
; do \
	if [ ! -z "$dir" ]; then \
		rm -rf "build/$dir"; \
	fi; \
done
cd install && rm -rf -- *
user@x230-master:~/heads$ rm -rf crossgcc/ build/*
user@x230-master:~/heads$ tar zxvf archive.tar.gz ./
./packages/libgcrypt-1.8.3.tar.bz2
./packages/libassuan-2.5.1.tar.bz2
./packages/libassuan-2.5.3.tar.bz2
./packages/cryptsetup-1.7.5.tar.xz
./packages/cryptsetup-1.7.3.tar.xz
./packages/musl-1.1.15.tar.gz
./packages/linux-5.4.30.tar.xz
./packages/cairo-1.14.12.tar.xz
./packages/LVM2.2.02.168.tgz
./packages/coreboot-4.12.tar.xz
./packages/gnupg-2.2.20.tar.bz2
./packages/dropbear-2016.74.tar.bz2
./packages/busybox-1.28.0.tar.bz2
./packages/libpng-1.6.34.tar.gz
./packages/libusb-1.0.21.tar.bz2
./packages/gnupg-2.2.21.tar.bz2
./packages/linux-4.19.114.tar.xz
./packages/popt-1.16.tar.gz
./packages/libksba-1.3.5.tar.bz2
./packages/purism-blobs-f53d4074a81c70352d39839884caac20181274d1.tar.gz
./packages/linux-b87b58e1b057a2706d422fbdc76aa34309c6c90b.tar.xz
./packages/coreboot-4.8.1.tar.xz
./packages/mbedtls-2.4.2-gpl.tgz
./packages/libgcrypt-1.8.4.tar.bz2
./packages/coreboot-blobs-4.12.tar.xz
./packages/linux-4.19.63.tar.xz
./packages/libgpg-error-1.39.tar.bz2
./packages/libksba-1.4.0.tar.bz2
./packages/libgcrypt-1.8.6.tar.bz2
./packages/libgcrypt-1.8.5.tar.bz2
./packages/slang-2.3.1a.tar.bz2
./packages/zlib-1.2.11.tar.gz
./packages/libgpg-error-1.32.tar.bz2
./packages/libgpg-error-1.37.tar.bz2
./packages/pixman-0.34.0.tar.gz
./packages/linux-4.14.62.tar.xz
./packages/gnupg-2.2.10.tar.bz2
./packages/coreboot-blobs-4.8.1.tar.xz
./packages/util-linux-2.29.2.tar.xz
./packages/cryptsetup-2.3.3.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/acpica-unix2-20200110.tar.gz
./build/coreboot-4.12/util/crossgcc/tarballs/gmp-6.1.2.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/binutils-2.33.1.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/gcc-8.3.0.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/mpfr-4.0.2.tar.xz
./build/coreboot-4.12/util/crossgcc/tarballs/mpc-1.1.0.tar.gz
./build/coreboot-4.8.1/util/crossgcc/tarballs/acpica-unix2-20180531.tar.gz
user@x230-master:~/heads$ 
user@x230-master:~/heads$ 
user@x230-master:~/heads$ make BOARD=x230 CPUS=2 
2020-11-13 13:07:06-05:00 Wrong gawk detected: 4.2.1
Makefile:202: warning: overriding recipe for target 'all'
Makefile:68: warning: ignoring old recipe for target 'all'
tar xf "/home/user/heads/packages/gawk-4.2.1.tar.xz" -C "/home/user/heads/build"
touch "/home/user/heads/build/gawk-4.2.1/.extract"
touch "/home/user/heads/build/gawk-4.2.1/.patch"
cd "/home/user/heads/build/gawk-4.2.1/" ; \
./configure 2>&1 \
| tee "/home/user/heads/build/log/gawk.configure.log" \
> /dev/null
touch "/home/user/heads/build/gawk-4.2.1/.configured"
make -C "/home/user/heads/build/gawk-4.2.1/"  \
	2>&1 \
	| tee "/home/user/heads/build/log/gawk.log" \
	> /dev/null

[...]

@Thrilleratplay @explit7 :

  • Successful clean build of x230 board on QubesOS debian-10 based AppVM:
2020-11-14 23:10:27-05:00 MAKE coreboot
2020-11-14 23:11:03-05:00 DONE coreboot
"/home/user/heads/build/coreboot-4.8.1/x230/cbfstool" "/home/user/heads/build/coreboot-4.8.1/x230/coreboot.rom" print
Name                           Offset     Type           Size   Comp
cbfs master header             0x0        cbfs header        32 none
fallback/romstage              0x80       stage           85188 none
cpu_microcode_blob.bin         0x14dc0    microcode       25600 none
fallback/ramstage              0x1b240    stage           81881 none
config                         0x2f280    raw               589 none
revision                       0x2f540    raw               581 none
cmos.default                   0x2f7c0    cmos_default      256 none
cmos_layout.bin                0x2f900    cmos_layout      1804 none
fallback/dsdt.aml              0x30080    raw             13646 none
fallback/payload               0x33640    simple elf    6850500 none
(empty)                        0x6bbe40   null           144792 none
bootblock                      0x6df400   bootblock        3000 none
2020-11-14 23:11:03-05:00 INSTALL   build/coreboot-4.8.1/x230/coreboot.rom => build/x230/heads-x230-v0.2.0-949-g5614525.rom
2020-11-14 23:11:03-05:00 UNCHANGED build/coreboot-4.8.1/x230/coreboot.rom
93bcf58c0159439ed7e61d71e2809a1619ef073280408657ae2e0ff9dad6ead1  build/x230/heads-x230-v0.2.0-949-g5614525.rom
4b3be2b552ab6b60df7291bfbd819d9b92f88641662472e0ebd8ca7fc27a4c63  /home/user/heads/build/gawk-4.2.1/gawk

@tlaurion
Copy link
Collaborator

tlaurion commented Nov 15, 2020

My question is more: what are the differences between docker 10, QubesOS debian-10 Template and native debian-10 installation to cause the issue.

As specified under #571, I think its time to move out of trying to build on all OSes of the world, and go docker, where that docker should probably be guix-buildstack as @daym is currently working on, so that end user instructions are clear, implementable in CIs, limiting impacts of host differences and permitting to replicate the results of today's commit in 5 years from now, pulling the same buildstack revision and permitting to fixate toolstack revisions desired for different coreboot versions, for example.

Can't replicate :/

@tlaurion
Copy link
Collaborator

Interestingly enough, that was replicated today for the first time under CircleCI Ubuntu 24.04 docker image (with a lot of patches against coreboot 4.8.1 to try to keep some boards alive) over #1038

Happened against qemu-coreboot CircleCI build

tail /root/project/build/log/coreboot.log
-----
cp /root/project/build/coreboot-4.8.1/util/sconfig/sconfig.tab.h_shipped qemu-coreboot/util/sconfig/sconfig.tab.h
mkdir -p qemu-coreboot/util/sconfig/
cp /root/project/build/coreboot-4.8.1/util/sconfig/sconfig.tab.c_shipped qemu-coreboot/util/sconfig/sconfig.tab.c
printf "    HOSTCC     util/sconfig/lex.yy.o\n"
    HOSTCC     util/sconfig/lex.yy.o
gcc -I/root/project/build/coreboot-4.8.1/util/sconfig -Iqemu-coreboot/util/sconfig -g -fms-extensions -c -o qemu-coreboot/util/sconfig/lex.yy.o qemu-coreboot/util/sconfig/lex.yy.c
printf "    HOSTCC     util/sconfig/sconfig.tab.o\n"
    HOSTCC     util/sconfig/sconfig.tab.o
gcc -I/root/project/build/coreboot-4.8.1/util/sconfig -Iqemu-coreboot/util/sconfig -g -fms-extensions -c -o qemu-coreboot/util/sconfig/sconfig.tab.o qemu-coreboot/util/sconfig/sconfig.tab.c
touch qemu-coreboot/util/sconfig/.generated
printf "    HOSTCC     util/sconfig/main.o\n"
    HOSTCC     util/sconfig/main.o
gcc -I/root/project/build/coreboot-4.8.1/util/sconfig -Iqemu-coreboot/util/sconfig -g -fms-extensions -c -o qemu-coreboot/util/sconfig/main.o util/sconfig/main.c
printf "    HOSTCC     util/sconfig/sconfig (link)\n"
    HOSTCC     util/sconfig/sconfig (link)
gcc -I/root/project/build/coreboot-4.8.1/util/sconfig -Iqemu-coreboot/util/sconfig -o qemu-coreboot/util/sconfig/sconfig qemu-coreboot/util/sconfig/lex.yy.o qemu-coreboot/util/sconfig/sconfig.tab.o qemu-coreboot/util/sconfig/main.o
/usr/bin/ld: qemu-coreboot/util/sconfig/main.o:/root/project/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: multiple definition of `head'; qemu-coreboot/util/sconfig/sconfig.tab.o:/root/project/build/coreboot-4.8.1/util/sconfig/sconfig.h:80: first defined here
collect2: error: ld returned 1 exit status
make[2]: *** [util/sconfig/Makefile.inc:43: qemu-coreboot/util/sconfig/sconfig] Error 1
make[2]: Leaving directory '/root/project/build/coreboot-4.8.1'
make[1]: *** [Makefile:455: /root/project/build/coreboot-4.8.1/qemu-coreboot/.build] Error 1
make[1]: Leaving directory '/root/project'
make: *** [Makefile:668: all] Error 2

@pgeorgi
Copy link

pgeorgi commented Oct 15, 2021

Make that line in sconfig.h:80 state extern struct device *head;

@pgeorgi
Copy link

pgeorgi commented Oct 22, 2021

I put up the change for 4.8_branch at https://review.coreboot.org/c/coreboot/+/58534

@tlaurion
Copy link
Collaborator

Thanks @pgeorgi

@tlaurion
Copy link
Collaborator

tlaurion commented May 10, 2024

I think those errors linked to buildstack and all others are now solved with #1661

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants