Completion generation with `meson` requires valid subgid/subuid #1246

Foxboron · 2023-02-26T14:51:41Z

Describe the bug
During building meson will try to create completions and this fails as the user doesn't have a set subuid/subgid in the build environment.

Steps how to reproduce the behaviour
Build toolbox without a subuid/subgid for the user.

Expected behaviour
Building toolbox in a restrictive build environment should work.

Actual behaviour
Building toolbox in a restrictive build environment doesn't work.

Screenshots
If applicable, add screenshots to help explain your problem.

Output of toolbox --version (v0.0.90+)
Latest toolbox release.

The text was updated successfully, but these errors were encountered:

Foxboron · 2023-02-26T15:06:26Z

I think we can do something like this.

λ src main» git diff cmd/completion.go
diff --git a/src/cmd/completion.go b/src/cmd/completion.go
index 26ed378..c71135b 100644
--- a/src/cmd/completion.go
+++ b/src/cmd/completion.go
@@ -32,6 +32,7 @@ var completionCmd = &cobra.Command{
        ValidArgs:             []string{"bash", "fish", "zsh"},
        Args:                  cobra.ExactValidArgs(1),
        RunE:                  completion,
+       PersistentPreRunE:     func(cmd *cobra.Command, args []string) error { return nil },
 }

 func init() {

It would ignore some of the setup done with regard to the configuration and logging, but do we need that for the completion sub command?

antonc42 · 2023-02-27T15:03:57Z

I'm building toolbox with meson in a Python virtualenv and getting this error. Is it related to this bug? Is there a workaround? I'm not familiar with subuid/subgid.

meson setup -Dprofile_dir=/etc/profile.d builddir
The Meson build system
Version: 1.0.1
Source dir: /home/user/scripts/update.d/toolbox-build/0.0.99.4
Build dir: /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir
Build type: native build
Project name: toolbox
Project version: 0.0.99.4
C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
C linker for the host machine: cc ld.bfd 2.34
Host machine cpu family: x86_64
Host machine cpu: x86_64
Compiler for C supports arguments -print-file-name=libc.so: YES 

meson.build:17:0: ERROR: C header 'shadow/subid.h' not found

A full log can be found at /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-logs/meson-log.txt

Contents of the log file:

Build started at 2023-02-27T08:53:54.119687
Main binary: /home/user/scripts/update.d/venv/bin/python3
Build Options: -Dprofile_dir=/etc/profile.d
Python system: Linux
The Meson build system
Version: 1.0.1
Source dir: /home/user/scripts/update.d/toolbox-build/0.0.99.4
Build dir: /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir
Build type: native build
Project name: toolbox
Project version: 0.0.99.4
-----
Detecting compiler via: cc --version
compiler returned <subprocess.Popen object at 0x7f25c8adc760>
compiler stdout:
cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


compiler stderr:

Running command: cc -E -dM -
-----
Detecting linker via: cc -Wl,--version
linker returned <subprocess.Popen object at 0x7f25c8a841f0>
linker stdout:
GNU ld (GNU Binutils for Ubuntu) 2.34
Copyright (C) 2020 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 9.4.0
/usr/bin/ld -plugin /usr/lib/gcc/x86_64-linux-gnu/9/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper -plugin-opt=-fresolution=/tmp/ccPCscQX.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -z relro /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/9/../../.. --version -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crtn.o

Sanity testing C compiler: cc
Is cross compiler: False.
Sanity check compiler command line: cc sanitycheckc.c -o sanitycheckc.exe -D_FILE_OFFSET_BITS=64
Sanity check compile stdout:

-----
Sanity check compile stderr:

-----
Running test binary command:  /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/sanitycheckc.exe
C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
C linker for the host machine: cc ld.bfd 2.34
-----
Detecting compiler via: cc --version
compiler returned <subprocess.Popen object at 0x7f25c8adcb80>
compiler stdout:
cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


compiler stderr:

Running command: cc -E -dM -
-----
Detecting linker via: cc -Wl,--version
linker returned <subprocess.Popen object at 0x7f25c8a463d0>
linker stdout:
GNU ld (GNU Binutils for Ubuntu) 2.34
Copyright (C) 2020 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

linker stderr:
collect2 version 9.4.0
/usr/bin/ld -plugin /usr/lib/gcc/x86_64-linux-gnu/9/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper -plugin-opt=-fresolution=/tmp/ccv28i5o.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -z relro /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/Scrt1.o /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/9/crtbeginS.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/9/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/9/../../.. --version -lgcc --push-state --as-needed -lgcc_s --pop-state -lc -lgcc --push-state --as-needed -lgcc_s --pop-state /usr/lib/gcc/x86_64-linux-gnu/9/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/crtn.o

Sanity testing C compiler: cc
Is cross compiler: False.
Sanity check compiler command line: cc sanitycheckc.c -o sanitycheckc.exe -D_FILE_OFFSET_BITS=64
Sanity check compile stdout:

-----
Sanity check compile stderr:

-----
Running test binary command:  /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/sanitycheckc.exe
C compiler for the build machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0")
C linker for the build machine: cc ld.bfd 2.34
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Running compile:
Working directory:  /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpfde2cs25
Command line:  cc /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpfde2cs25/testfile.c -o /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpfde2cs25/output.obj -c -D_FILE_OFFSET_BITS=64 -O0 -Wl,--start-group -print-file-name=libc.so -Wl,--end-group 

Code:
 extern int i;
int i;

Compiler stdout:
 /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libc.so

Compiler stderr:
 
Compiler for C supports arguments -print-file-name=libc.so: YES 
Running compile:
Working directory:  /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpph6_a0qc
Command line:  cc /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpph6_a0qc/testfile.c -E -P -D_FILE_OFFSET_BITS=64 -P -O0 -std=c99 

Code:
 
        #ifdef __has_include
         #if !__has_include("shadow/subid.h")
          #error "Header 'shadow/subid.h' could not be found"
         #endif
        #else
         #include <shadow/subid.h>
        #endif
Compiler stdout:
 
Compiler stderr:
 /home/user/scripts/update.d/toolbox-build/0.0.99.4/builddir/meson-private/tmpph6_a0qc/testfile.c:4:12: error: #error "Header 'shadow/subid.h' could not be found"
    4 |           #error "Header 'shadow/subid.h' could not be found"
      |            ^~~~~


meson.build:17:0: ERROR: C header 'shadow/subid.h' not found

Foxboron · 2023-02-27T15:07:18Z

That error is not relevant for this bug. You are missing headers on your system.

debarshiray · 2023-03-01T18:17:28Z

I'm building toolbox with meson in a Python virtualenv and getting this error. Is it related to this bug? Is there a workaround? I'm not familiar with subuid/subgid.

[...]
meson.build:17:0: ERROR: C header 'shadow/subid.h' not found

As @Foxboron said, you are missing some header files and such. Shadow 4.9 introduced a new shared library called libsubid.so and shadow/subid.h is the corresponding header file.

debarshiray · 2023-03-01T18:19:25Z

@Foxboron did you notice #1248 ?

I encountered a similar, but not exactly the same, problem when building Toolbx 0.0.99.4 for Fedora.

Foxboron · 2023-03-01T18:20:24Z

@debarshiray I didn't! I'll test it in a day or two :)

antonc42 · 2023-03-01T18:37:09Z

I'm building toolbox with meson in a Python virtualenv and getting this error. Is it related to this bug? Is there a workaround? I'm not familiar with subuid/subgid.
[...]
meson.build:17:0: ERROR: C header 'shadow/subid.h' not found

As @Foxboron said, you are missing some header files and such. Shadow 4.9 introduced a new shared library called libsubid.so and shadow/subid.h is the corresponding header file.

@debarshiray Thanks. I opened a separate bug report: #1247

Having a separate convenience function reduces the indentation levels by at least one, and sometimes two, and makes it easy to have more detailed debug logs. This will make the subsequent commit easier to read. containers#1246

Ever since commit bafbbe8, the shell completions are generated while building Toolbx using the 'completion' command. This involves running toolbox(1) itself, and hence validating the subordinate user and group ID ranges. Unfortunately, some build environments don't have subordinate ID ranges set up. Therefore, it's better to avoid validating them when generating the shell completions, especially, since they are generated by Cobra itself and subordinate ID ranges are not involved at all. Note that subordinate ID ranges may be needed when the generated shell completions are actually used in interactive command line environments. The shell completions invoke the hidden '__complete' command to get the results that are presented to the user, and, if needed, the subordinate ID ranges will continue to be used by podman(1) as part of that. containers#1246

Ever since commit bafbbe8, the shell completions are generated while building Toolbx using the 'completion' command. This involves running toolbox(1) itself, and hence validating the subordinate user and group ID ranges. Unfortunately, some build environments don't have subordinate ID ranges set up. Therefore, it's better to avoid validating them when generating the shell completions, especially, since they are generated by Cobra itself and subordinate ID ranges are not involved at all. Note that subordinate ID ranges may be needed when the generated shell completions are actually used in interactive command line environments. The shell completions invoke the hidden '__complete' command to get the results that are presented to the user, and, if needed, the subordinate ID ranges will continue to be used by podman(1) as part of that. Some changes by Debarshi Ray. containers#1246

Ever since commit bafbbe8, the shell completions are generated while building Toolbx using the 'completion' command. This involves running toolbox(1) itself, and hence validating the subordinate user and group ID ranges. Unfortunately, some build environments, like openSUSE's, don't have subordinate ID ranges set up. Therefore, it's better to not validate the subordinate IDs ranges when generating the shell completions, since they are generated by Cobra itself and subordinate ID ranges are not involved at all. Note that subordinate ID ranges may be needed when the generated shell completions are actually used in interactive command line environments. The shell completions invoke the hidden '__complete' command to get the results that are presented to the user, and, if needed, the subordinate ID ranges will continue to be used by podman(1) as part of that. Some changes by Debarshi Ray. containers#1246

This is meant to roughly replicate the build environments used by downstream distributors to build toolbox(1). These can be restricted in odd ways compared to a fully featured environment where toolbox(1) is actually going to be used. It's important to ensure that toolbox(1) can be built by downstream distributors without any unnecessary hassle. containers/podman#17657 containers#1246

Ever since commit bafbbe8, the shell completions are generated while building Toolbx using the 'completion' command. This involves running toolbox(1) itself, and hence validating the subordinate user and group ID ranges. Unfortunately, some build environments, like openSUSE's, don't have subordinate ID ranges set up. Therefore, it's better to not validate the subordinate IDs ranges when generating the shell completions, since they are generated by Cobra itself and subordinate ID ranges are not involved at all. Note that subordinate ID ranges may be needed when the generated shell completions are actually used in interactive command line environments. The shell completions invoke the hidden '__complete' command to get the results that are presented to the user, and, if needed, the subordinate ID ranges will continue to be used by podman(1) as part of that. Some changes by Debarshi Ray. containers#1246 containers#1249

This is meant to roughly replicate the build environments used by downstream distributors to build toolbox(1). These can be restricted in odd ways compared to a fully featured environment where toolbox(1) is actually going to be used. It's important to ensure that toolbox(1) can be built by downstream distributors without any unnecessary hassle. containers/podman#17657 containers#1246

debarshiray · 2023-03-01T20:55:49Z

@debarshiray I didn't! I'll test it in a day or two :)

The problem that you reported here is different from the one I experienced for Fedora. Turns out that folks at openSUSE are also having the same problem as you.

Anyway, I attempted a fix at #1252 and added a test so that hopefully this doesn't regress.

Ever since commit bafbbe8, the shell completions are generated while building Toolbx using the 'completion' command. This involves running toolbox(1) itself, and hence validating the subordinate user and group ID ranges. Unfortunately, some build environments, like openSUSE's, don't have subordinate ID ranges set up. Therefore, it's better to not validate the subordinate ID ranges when generating the shell completions, since they are generated by Cobra itself and subordinate ID ranges are not involved at all. Note that subordinate ID ranges may be needed when the generated shell completions are actually used in interactive command line environments. The shell completions invoke the hidden '__complete' command to get the results that are presented to the user, and, if needed, the subordinate ID ranges will continue to be used by podman(1) as part of that. Some changes by Debarshi Ray. containers#1246 containers#1249

This is meant to roughly replicate the build environments used by downstream distributors to build toolbox(1). These can be restricted in odd ways compared to a fully featured environment where toolbox(1) is actually going to be used. eg., the inability to use podman(1) in the case of Fedora or not having subordinate user and group ID ranges in the case of openSUSE. It's important to ensure that toolbox(1) can be built by downstream distributors without any unnecessary hassle. containers/podman#17657 containers#1246

Having a separate convenience function reduces the indentation levels by at least one, and sometimes two, and makes it easy to have more detailed debug logs. This will make the subsequent commit easier to read. containers#1246

Ever since commit bafbbe8, the shell completions are generated while building Toolbx using the 'completion' command. This involves running toolbox(1) itself, and hence validating the subordinate user and group ID ranges. Unfortunately, some build environments, like openSUSE's, don't have subordinate ID ranges set up. Therefore, it's better to not validate the subordinate ID ranges when generating the shell completions, since they are generated by Cobra itself and subordinate ID ranges are not involved at all. Note that subordinate ID ranges may be needed when the generated shell completions are actually used in interactive command line environments. The shell completions invoke the hidden '__complete' command to get the results that are presented to the user, and, if needed, the subordinate ID ranges will continue to be used by podman(1) as part of that. Some changes by Debarshi Ray. containers#1246 containers#1249

This is meant to roughly replicate the build environments used by downstream distributors to build toolbox(1). These can be restricted in odd ways compared to a fully featured environment where toolbox(1) is actually going to be used. eg., the inability to use podman(1) in the case of Fedora or not having subordinate user and group ID ranges in the case of openSUSE. It's important to ensure that toolbox(1) can be built by downstream distributors without any unnecessary hassle. containers/podman#17657 containers#1246

debarshiray · 2023-03-02T10:42:04Z

I am closing this because I believe this is fixed by #1252 now. Please re-open if you have reason to believe otherwise. :)

Foxboron · 2023-03-02T19:52:19Z

@debarshiray Everything works after applying all the three relevant commits.

816a7ca
f555029
fc5f568

debarshiray · 2023-03-08T20:45:13Z

@debarshiray Everything works after applying all the three relevant commits.

Yay! Thanks for the feedback.

Foxboron added the 1. Bug Something isn't working label Feb 26, 2023

debarshiray mentioned this issue Mar 1, 2023

cmd/root: Don't validate subordinate IDs when generating the completions #1252

Merged

debarshiray closed this as completed Mar 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Completion generation with `meson` requires valid subgid/subuid #1246

Completion generation with `meson` requires valid subgid/subuid #1246

Foxboron commented Feb 26, 2023

Foxboron commented Feb 26, 2023

antonc42 commented Feb 27, 2023

Foxboron commented Feb 27, 2023

debarshiray commented Mar 1, 2023

debarshiray commented Mar 1, 2023

Foxboron commented Mar 1, 2023

antonc42 commented Mar 1, 2023

debarshiray commented Mar 1, 2023

debarshiray commented Mar 2, 2023

Foxboron commented Mar 2, 2023

debarshiray commented Mar 8, 2023

Completion generation with meson requires valid subgid/subuid #1246

Completion generation with meson requires valid subgid/subuid #1246

Comments

Foxboron commented Feb 26, 2023

Foxboron commented Feb 26, 2023

antonc42 commented Feb 27, 2023

Foxboron commented Feb 27, 2023

debarshiray commented Mar 1, 2023

debarshiray commented Mar 1, 2023

Foxboron commented Mar 1, 2023

antonc42 commented Mar 1, 2023

debarshiray commented Mar 1, 2023

debarshiray commented Mar 2, 2023

Foxboron commented Mar 2, 2023

debarshiray commented Mar 8, 2023

Completion generation with `meson` requires valid subgid/subuid #1246

Completion generation with `meson` requires valid subgid/subuid #1246