Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_psgi_hot_deploy_with_reload is failing on RHEL8 w/o OpenShift #280

Closed
phracek opened this issue Aug 30, 2023 · 20 comments
Closed

test_psgi_hot_deploy_with_reload is failing on RHEL8 w/o OpenShift #280

phracek opened this issue Aug 30, 2023 · 20 comments
Assignees

Comments

@phracek
Copy link
Member

phracek commented Aug 30, 2023

Container platform

Podman/Docker

Version

Perl 5.26

OS version of the container image

RHEL 8

Bugzilla, Jira

No response

Description

See output log from our nightly build script:

Running test test_psgi_hot_deploy_with_reload (starting at 2023-08-29 22:40:40-04:00) ... 
-----------------------------------------------

�[1m[INFO] Starting tests for psgi-hot_deploy....�[0m


�[1m[INFO] Build the test application image...�[0m

hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint: 	git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint: 	git branch -m <name>
Initialized empty Git repository in /var/tmp/daily_scl_tests/rhel8-test/plans/nightly-container-rhel8/data/s2i-perl-container/test/psgi-hot_deploy/.git/
[master (root-commit) c8a0add] Sample commit
 3 files changed, 38 insertions(+)
 create mode 100644 cpanfile
 create mode 100755 index.psgi
 create mode 100644 lib/My/Test.pm
STEP 1/12: FROM ubi8/perl-526:1
STEP 2/12: LABEL "io.openshift.s2i.build.image"="ubi8/perl-526:1"       "io.openshift.s2i.build.source-location"="file:///var/tmp/daily_scl_tests/rhel8-test/plans/nightly-container-rhel8/data/s2i-perl-container/5.26-mod_fcgid/test/psgi-hot_deploy"
--> dc163130b43
STEP 3/12: USER root
--> 57ca068cb39
STEP 4/12: COPY upload/src/ /tmp/src
--> 5485575375e
STEP 5/12: RUN chown -R 1001:0 /tmp/src
--> 4ae1fb2d547
STEP 6/12: ENV NPM_MIRROR=https://repository.engineering.redhat.com/nexus/repository/registry.npmjs.org
--> 515cb247320
STEP 7/12: ENV PSGI_FILE=./index.psgi
--> 8b6191b438a
STEP 8/12: ENV PSGI_RELOAD=1
--> f6f447bbbc5
STEP 9/12: RUN cd /etc/pki/ca-trust/source/anchors && update-ca-trust extract
--> e0fd8c81a77
STEP 10/12: USER 1001
--> bc25b47748f
STEP 11/12: RUN /usr/libexec/s2i/assemble
---> Installing application source ...
---> PSGI application found in ./index.psgi
---> Installing modules from cpanfile ...
--> Working on .
Configuring /opt/app-root/src ... OK
==> Found dependencies: Plack::Handler::FCGI, FCGI::ProcManager
--> Working on Plack::Handler::FCGI
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Plack-1.0050.tar.gz ... OK
==> Found dependencies: File::ShareDir::Install
--> Working on File::ShareDir::Install
Fetching http://www.cpan.org/authors/id/E/ET/ETHER/File-ShareDir-Install-0.14.tar.gz ... OK
Configuring File-ShareDir-Install-0.14 ... OK
Building File-ShareDir-Install-0.14 ... OK
Successfully installed File-ShareDir-Install-0.14
Configuring Plack-1.0050 ... OK
==> Found dependencies: Cookie::Baker, WWW::Form::UrlEncoded, Devel::StackTrace, Devel::StackTrace::AsHTML, Filesys::Notify::Simple, HTTP::Entity::Parser, HTTP::Headers::Fast, Test::TCP, Hash::MultiValue, Apache::LogFormat::Compiler, HTTP::Message, File::ShareDir, Stream::Buffered, Try::Tiny
--> Working on Cookie::Baker
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/Cookie-Baker-0.11.tar.gz ... OK
==> Found dependencies: Module::Build::Tiny
--> Working on Module::Build::Tiny
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/Module-Build-Tiny-0.046.tar.gz ... OK
==> Found dependencies: ExtUtils::Helpers, ExtUtils::InstallPaths, ExtUtils::Config
--> Working on ExtUtils::Helpers
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Helpers-0.026.tar.gz ... OK
Configuring ExtUtils-Helpers-0.026 ... OK
Building ExtUtils-Helpers-0.026 ... OK
Successfully installed ExtUtils-Helpers-0.026
--> Working on ExtUtils::InstallPaths
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-InstallPaths-0.012.tar.gz ... OK
Configuring ExtUtils-InstallPaths-0.012 ... OK
==> Found dependencies: ExtUtils::Config
--> Working on ExtUtils::Config
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Config-0.008.tar.gz ... OK
Configuring ExtUtils-Config-0.008 ... OK
Building ExtUtils-Config-0.008 ... OK
Successfully installed ExtUtils-Config-0.008
Building ExtUtils-InstallPaths-0.012 ... OK
Successfully installed ExtUtils-InstallPaths-0.012
Configuring Module-Build-Tiny-0.046 ... OK
Building Module-Build-Tiny-0.046 ... OK
Successfully installed Module-Build-Tiny-0.046
Configuring Cookie-Baker-0.11 ... OK
Building Cookie-Baker-0.11 ... OK
Successfully installed Cookie-Baker-0.11
--> Working on WWW::Form::UrlEncoded
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/WWW-Form-UrlEncoded-0.26.tar.gz ... OK
Configuring WWW-Form-UrlEncoded-0.26 ... OK
Building WWW-Form-UrlEncoded-0.26 ... OK
Successfully installed WWW-Form-UrlEncoded-0.26
--> Working on Devel::StackTrace
Fetching http://www.cpan.org/authors/id/D/DR/DROLSKY/Devel-StackTrace-2.04.tar.gz ... OK
Configuring Devel-StackTrace-2.04 ... OK
Building Devel-StackTrace-2.04 ... OK
Successfully installed Devel-StackTrace-2.04
--> Working on Devel::StackTrace::AsHTML
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Devel-StackTrace-AsHTML-0.15.tar.gz ... OK
Configuring Devel-StackTrace-AsHTML-0.15 ... OK
Building Devel-StackTrace-AsHTML-0.15 ... OK
Successfully installed Devel-StackTrace-AsHTML-0.15
--> Working on Filesys::Notify::Simple
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Filesys-Notify-Simple-0.14.tar.gz ... OK
Configuring Filesys-Notify-Simple-0.14 ... OK
Building Filesys-Notify-Simple-0.14 ... OK
Successfully installed Filesys-Notify-Simple-0.14
--> Working on HTTP::Entity::Parser
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/HTTP-Entity-Parser-0.25.tar.gz ... OK
Configuring HTTP-Entity-Parser-0.25 ... OK
==> Found dependencies: HTTP::MultiPartParser, Stream::Buffered, Hash::MultiValue, JSON::MaybeXS
--> Working on HTTP::MultiPartParser
Fetching http://www.cpan.org/authors/id/C/CH/CHANSEN/HTTP-MultiPartParser-0.02.tar.gz ... OK
Configuring HTTP-MultiPartParser-0.02 ... OK
==> Found dependencies: Test::Deep
--> Working on Test::Deep
Fetching http://www.cpan.org/authors/id/R/RJ/RJBS/Test-Deep-1.204.tar.gz ... OK
Configuring Test-Deep-1.204 ... OK
Building Test-Deep-1.204 ... OK
Successfully installed Test-Deep-1.204
Building HTTP-MultiPartParser-0.02 ... OK
Successfully installed HTTP-MultiPartParser-0.02
--> Working on Stream::Buffered
Fetching http://www.cpan.org/authors/id/D/DO/DOY/Stream-Buffered-0.03.tar.gz ... OK
Configuring Stream-Buffered-0.03 ... OK
Building Stream-Buffered-0.03 ... OK
Successfully installed Stream-Buffered-0.03
--> Working on Hash::MultiValue
Fetching http://www.cpan.org/authors/id/A/AR/ARISTOTLE/Hash-MultiValue-0.16.tar.gz ... OK
Configuring Hash-MultiValue-0.16 ... OK
Building Hash-MultiValue-0.16 ... OK
Successfully installed Hash-MultiValue-0.16
--> Working on JSON::MaybeXS
Fetching http://www.cpan.org/authors/id/E/ET/ETHER/JSON-MaybeXS-1.004005.tar.gz ... OK
Configuring JSON-MaybeXS-1.004005 ... OK
==> Found dependencies: Cpanel::JSON::XS
--> Working on Cpanel::JSON::XS
Fetching http://www.cpan.org/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-4.37.tar.gz ... OK
Configuring Cpanel-JSON-XS-4.37 ... OK
Building Cpanel-JSON-XS-4.37 ... OK
Successfully installed Cpanel-JSON-XS-4.37
Building JSON-MaybeXS-1.004005 ... OK
Successfully installed JSON-MaybeXS-1.004005
Building HTTP-Entity-Parser-0.25 ... OK
Successfully installed HTTP-Entity-Parser-0.25
--> Working on HTTP::Headers::Fast
Fetching http://www.cpan.org/authors/id/T/TO/TOKUHIROM/HTTP-Headers-Fast-0.22.tar.gz ... OK
Configuring HTTP-Headers-Fast-0.22 ... OK
==> Found dependencies: HTTP::Date
--> Working on HTTP::Date
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/HTTP-Date-6.06.tar.gz ... OK
Configuring HTTP-Date-6.06 ... OK
==> Found dependencies: Time::Zone
--> Working on Time::Zone
Fetching http://www.cpan.org/authors/id/A/AT/ATOOMIC/TimeDate-2.33.tar.gz ... OK
Configuring TimeDate-2.33 ... OK
Building TimeDate-2.33 ... OK
Successfully installed TimeDate-2.33
Building HTTP-Date-6.06 ... OK
Successfully installed HTTP-Date-6.06
Building HTTP-Headers-Fast-0.22 ... OK
Successfully installed HTTP-Headers-Fast-0.22
--> Working on Test::TCP
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Test-TCP-2.22.tar.gz ... OK
Configuring Test-TCP-2.22 ... OK
==> Found dependencies: Test::SharedFork
--> Working on Test::SharedFork
Fetching http://www.cpan.org/authors/id/E/EX/EXODIST/Test-SharedFork-0.35.tar.gz ... OK
Configuring Test-SharedFork-0.35 ... OK
Building Test-SharedFork-0.35 ... OK
Successfully installed Test-SharedFork-0.35
Building Test-TCP-2.22 ... OK
Successfully installed Test-TCP-2.22
--> Working on Apache::LogFormat::Compiler
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/Apache-LogFormat-Compiler-0.36.tar.gz ... OK
Configuring Apache-LogFormat-Compiler-0.36 ... OK
==> Found dependencies: POSIX::strftime::Compiler
--> Working on POSIX::strftime::Compiler
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/POSIX-strftime-Compiler-0.44.tar.gz ... OK
Configuring POSIX-strftime-Compiler-0.44 ... OK
Building POSIX-strftime-Compiler-0.44 ... OK
Successfully installed POSIX-strftime-Compiler-0.44
Building Apache-LogFormat-Compiler-0.36 ... OK
Successfully installed Apache-LogFormat-Compiler-0.36
--> Working on HTTP::Message
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/HTTP-Message-6.44.tar.gz ... OK
Configuring HTTP-Message-6.44 ... OK
==> Found dependencies: Clone, IO::HTML, Encode, LWP::MediaTypes
--> Working on Clone
Fetching http://www.cpan.org/authors/id/G/GA/GARU/Clone-0.46.tar.gz ... OK
Configuring Clone-0.46 ... OK
Building Clone-0.46 ... OK
Successfully installed Clone-0.46
--> Working on IO::HTML
Fetching http://www.cpan.org/authors/id/C/CJ/CJM/IO-HTML-1.004.tar.gz ... OK
Configuring IO-HTML-1.004 ... OK
==> Found dependencies: Encode
--> Working on Encode
Fetching http://www.cpan.org/authors/id/D/DA/DANKOGAI/Encode-3.19.tar.gz ... OK
Configuring Encode-3.19 ... OK
Building Encode-3.19 ... OK
Successfully installed Encode-3.19 (upgraded from 2.97)
Building IO-HTML-1.004 ... OK
Successfully installed IO-HTML-1.004
--> Working on LWP::MediaTypes
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/LWP-MediaTypes-6.04.tar.gz ... OK
Configuring LWP-MediaTypes-6.04 ... OK
Building LWP-MediaTypes-6.04 ... OK
Successfully installed LWP-MediaTypes-6.04
Building HTTP-Message-6.44 ... OK
Successfully installed HTTP-Message-6.44
--> Working on File::ShareDir
Fetching http://www.cpan.org/authors/id/R/RE/REHSACK/File-ShareDir-1.118.tar.gz ... OK
Configuring File-ShareDir-1.118 ... OK
==> Found dependencies: Class::Inspector
--> Working on Class::Inspector
Fetching http://www.cpan.org/authors/id/P/PL/PLICEASE/Class-Inspector-1.36.tar.gz ... OK
Configuring Class-Inspector-1.36 ... OK
Building Class-Inspector-1.36 ... OK
Successfully installed Class-Inspector-1.36
Building File-ShareDir-1.118 ... OK
Successfully installed File-ShareDir-1.118
--> Working on Try::Tiny
Fetching http://www.cpan.org/authors/id/E/ET/ETHER/Try-Tiny-0.31.tar.gz ... OK
Configuring Try-Tiny-0.31 ... OK
Building Try-Tiny-0.31 ... OK
Successfully installed Try-Tiny-0.31
Building Plack-1.0050 ... OK
Successfully installed Plack-1.0050
--> Working on FCGI::ProcManager
Fetching http://www.cpan.org/authors/id/A/AR/ARODLAND/FCGI-ProcManager-0.28.tar.gz ... OK
Configuring FCGI-ProcManager-0.28 ... OK
Building FCGI-ProcManager-0.28 ... OK
Successfully installed FCGI-ProcManager-0.28
<== Installed dependencies for .. Finishing.
34 distributions installed
---> Patching Plack::Handler::FCGI to work with mod_fcgi ...
~/extlib/lib/perl5 ~
patching file Plack/Handler/FCGI.pm
~
--> 335b302ff8d
STEP 12/12: CMD /usr/libexec/s2i/run
COMMIT ubi8/perl-526:1-testapp
--> a85a5cefb9c
Successfully tagged localhost/ubi8/perl-526:1-testapp
a85a5cefb9c1ce3c458e9bceaf2710acdf2d76abd20aaa48be84fdc485c4ad8e
Waiting for container start... 1

�[1m[INFO] Testing the HTTP response for <http://10.88.1.124:8080/>...�[0m


�[1m[INFO] Testing the HTTP response for <http://10.88.1.124:8080/>...�[0m


�[1m[INFO] Changing source code of application...�[0m


�[1m[INFO] Testing the HTTP response for <http://10.88.1.124:8080/>...�[0m

Test for image 'ubi8/perl-526:1' FAILED (exit code: 1)
Test for image 'ubi8/perl-526:1' FAILED (exit code: 1)

�[1m[INFO] Cleaning up the test application...�[0m

afaa97737034fa10c156648214c7a03f6f416ff3509d29ec902f5bc8eb727e26
Error: no container with ID or name "afaa97737034fa10c156648214c7a03f6f416ff3509d29ec902f5bc8eb727e26" found: no such container
rm: cannot remove '/tmp/tmp.1XglKr28st/cid': No such file or directory

�[1m[INFO] Cleaning up the test application image...�[0m

Untagged: localhost/ubi8/perl-526:1-testapp
Deleted: a85a5cefb9c1ce3c458e9bceaf2710acdf2d76abd20aaa48be84fdc485c4ad8e
Deleted: 335b302ff8dc816ef2eb1fa7cca6c9ac9a0c04c81ad5ce8524bc42742ebcbc9b
Deleted: bc25b47748faadeebbc38ee259e77f7cce616e98c597a27e88df04c1f06512af
Deleted: e0fd8c81a775aca015b883e310a8e161d162dd7cc36cf5744a24c004b95585a3
Deleted: f6f447bbbc5356e293965453acdffae6d2ae3b03817424633a45dc9f6bf23f16
Deleted: 8b6191b438a11995a066e2c21f4cb3a2808300060863bf044796e67ffaeea4ce
Deleted: 515cb2473205dfb50718cfd764b4de3301974e805d3823a4e552bb31a12bb684
Deleted: 4ae1fb2d547a81d9bef9d4babc22eaba6c1834f0a5d5ca95f8ec20d6171bac19
Deleted: 5485575375e5fa6e79b54652d82673f4030155df26d803f3a11fa4a25b951ac6
Deleted: 57ca068cb3977aeb2a82136d2d527166966ca54df3db6b927913de24ad735459
Deleted: dc163130b432c6e0758ebbffed91d08823848de61e9ea47cb422198a66512b3b

�[1m[INFO] All tests for the psgi-hot_deploy finished successfully....�[0m

Reproducer

No response

@michal-josef-spacek
Copy link
Member

I will look to it

@michal-josef-spacek
Copy link
Member

@phracek Are you sure, that is still present?

make test TARGET=rhel8 VERSIONS=5.26-mod_fcgid
...
Tests were run for image ubi8/perl-526:1
==============================================
Test cases results:

 [PASSED] for 'appnamenotset' test_sample_test_app (00:00:10)
 [PASSED] for 'appnamenotset' test_binpath (00:00:05)
 [PASSED] for 'appnamenotset' test_psgi (00:00:57)
 [PASSED] for 'appnamenotset' test_psgi_hot_deploy_without_reload (00:00:53)
 [PASSED] for 'appnamenotset' test_psgi_hot_deploy_with_reload (00:00:54)
 [PASSED] for 'appnamenotset' test_psgi_variables (00:00:57)
 [PASSED] for 'appnamenotset' test_warningonstderr (00:00:04)
 [PASSED] for 'appnamenotset' test_fcgi (00:00:04)
 [PASSED] for 'appnamenotset' test_npm (00:00:06)
 [PASSED] for 'appnamenotset' test_from_dockerfile (00:03:33)

@michal-josef-spacek
Copy link
Member

REPOSITORY                                 TAG                IMAGE ID       CREATED          SIZE
ubi8/perl-526                              1                  c6c9f8e0b630   15 minutes ago   750MB
registry.access.redhat.com/ubi8/s2i-base   1                  32e00364c94c   2 weeks ago      655MB

@phracek
Copy link
Member Author

phracek commented Sep 11, 2023

Yes,

on RHEL8 host running command make test TARGET=rhel8 VERSIONS=5.26-mod_fcgid

-----------------------------------------------
Running test test_psgi_hot_deploy_with_reload (starting at 2023-09-11 08:19:50-04:00) ...
-----------------------------------------------

[INFO] Starting tests for psgi-hot_deploy....


[INFO] Build the test application image...

hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: 	git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: 	git branch -m <name>
Initialized empty Git repository in /root/s2i-perl-container/test/psgi-hot_deploy/.git/
fatal: not in a git directory
fatal: detected dubious ownership in repository at '/root/s2i-perl-container/test/psgi-hot_deploy'
To add an exception for this directory, call:

	git config --global --add safe.directory /root/s2i-perl-container/test/psgi-hot_deploy
STEP 1/12: FROM ubi8/perl-526:1
STEP 2/12: LABEL "io.openshift.s2i.build.image"="ubi8/perl-526:1"       "io.openshift.s2i.build.source-location"="file:///root/s2i-perl-container/5.26-mod_fcgid/test/psgi-hot_deploy"
--> 0f9368d5228
STEP 3/12: USER root
--> 9b9b1f3d693
STEP 4/12: COPY upload/src/ /tmp/src
--> 65845f69a33
STEP 5/12: RUN chown -R 1001:0 /tmp/src
--> f9fb29d5963
STEP 6/12: ENV NPM_MIRROR=https://repository.engineering.redhat.com/nexus/repository/registry.npmjs.org
--> 5401b0710b4
STEP 7/12: ENV PSGI_FILE=./index.psgi
--> 6fcb162653b
STEP 8/12: ENV PSGI_RELOAD=1
--> 085945e09a0
STEP 9/12: RUN cd /etc/pki/ca-trust/source/anchors && update-ca-trust extract
--> 23e46431160
STEP 10/12: USER 1001
--> 131db2da5ee
STEP 11/12: RUN /usr/libexec/s2i/assemble
---> Installing application source ...
---> PSGI application found in ./index.psgi
---> Installing modules from cpanfile ...
--> Working on .
Configuring /opt/app-root/src ... OK
==> Found dependencies: Plack::Handler::FCGI, FCGI::ProcManager
--> Working on Plack::Handler::FCGI
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Plack-1.0050.tar.gz ... OK
==> Found dependencies: File::ShareDir::Install
--> Working on File::ShareDir::Install
Fetching http://www.cpan.org/authors/id/E/ET/ETHER/File-ShareDir-Install-0.14.tar.gz ... OK
Configuring File-ShareDir-Install-0.14 ... OK
Building File-ShareDir-Install-0.14 ... OK
Successfully installed File-ShareDir-Install-0.14
Configuring Plack-1.0050 ... OK
==> Found dependencies: File::ShareDir, Cookie::Baker, Hash::MultiValue, Try::Tiny, WWW::Form::UrlEncoded, Test::TCP, Stream::Buffered, Devel::StackTrace, Devel::StackTrace::AsHTML, Apache::LogFormat::Compiler, Filesys::Notify::Simple, HTTP::Message, HTTP::Entity::Parser, HTTP::Headers::Fast
--> Working on File::ShareDir
Fetching http://www.cpan.org/authors/id/R/RE/REHSACK/File-ShareDir-1.118.tar.gz ... OK
Configuring File-ShareDir-1.118 ... OK
==> Found dependencies: Class::Inspector
--> Working on Class::Inspector
Fetching http://www.cpan.org/authors/id/P/PL/PLICEASE/Class-Inspector-1.36.tar.gz ... OK
Configuring Class-Inspector-1.36 ... OK
Building Class-Inspector-1.36 ... OK
Successfully installed Class-Inspector-1.36
Building File-ShareDir-1.118 ... OK
Successfully installed File-ShareDir-1.118
--> Working on Cookie::Baker
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/Cookie-Baker-0.11.tar.gz ... OK
==> Found dependencies: Module::Build::Tiny
--> Working on Module::Build::Tiny
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/Module-Build-Tiny-0.046.tar.gz ... OK
==> Found dependencies: ExtUtils::Helpers, ExtUtils::Config, ExtUtils::InstallPaths
--> Working on ExtUtils::Helpers
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Helpers-0.026.tar.gz ... OK
Configuring ExtUtils-Helpers-0.026 ... OK
Building ExtUtils-Helpers-0.026 ... OK
Successfully installed ExtUtils-Helpers-0.026
--> Working on ExtUtils::Config
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-Config-0.008.tar.gz ... OK
Configuring ExtUtils-Config-0.008 ... OK
Building ExtUtils-Config-0.008 ... OK
Successfully installed ExtUtils-Config-0.008
--> Working on ExtUtils::InstallPaths
Fetching http://www.cpan.org/authors/id/L/LE/LEONT/ExtUtils-InstallPaths-0.012.tar.gz ... OK
Configuring ExtUtils-InstallPaths-0.012 ... OK
Building ExtUtils-InstallPaths-0.012 ... OK
Successfully installed ExtUtils-InstallPaths-0.012
Configuring Module-Build-Tiny-0.046 ... OK
Building Module-Build-Tiny-0.046 ... OK
Successfully installed Module-Build-Tiny-0.046
Configuring Cookie-Baker-0.11 ... OK
Building Cookie-Baker-0.11 ... OK
Successfully installed Cookie-Baker-0.11
--> Working on Hash::MultiValue
Fetching http://www.cpan.org/authors/id/A/AR/ARISTOTLE/Hash-MultiValue-0.16.tar.gz ... OK
Configuring Hash-MultiValue-0.16 ... OK
Building Hash-MultiValue-0.16 ... OK
Successfully installed Hash-MultiValue-0.16
--> Working on Try::Tiny
Fetching http://www.cpan.org/authors/id/E/ET/ETHER/Try-Tiny-0.31.tar.gz ... OK
Configuring Try-Tiny-0.31 ... OK
Building Try-Tiny-0.31 ... OK
Successfully installed Try-Tiny-0.31
--> Working on WWW::Form::UrlEncoded
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/WWW-Form-UrlEncoded-0.26.tar.gz ... OK
Configuring WWW-Form-UrlEncoded-0.26 ... OK
Building WWW-Form-UrlEncoded-0.26 ... OK
Successfully installed WWW-Form-UrlEncoded-0.26
--> Working on Test::TCP
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Test-TCP-2.22.tar.gz ... OK
Configuring Test-TCP-2.22 ... OK
==> Found dependencies: Test::SharedFork
--> Working on Test::SharedFork
Fetching http://www.cpan.org/authors/id/E/EX/EXODIST/Test-SharedFork-0.35.tar.gz ... OK
Configuring Test-SharedFork-0.35 ... OK
Building Test-SharedFork-0.35 ... OK
Successfully installed Test-SharedFork-0.35
Building Test-TCP-2.22 ... OK
Successfully installed Test-TCP-2.22
--> Working on Stream::Buffered
Fetching http://www.cpan.org/authors/id/D/DO/DOY/Stream-Buffered-0.03.tar.gz ... OK
Configuring Stream-Buffered-0.03 ... OK
Building Stream-Buffered-0.03 ... OK
Successfully installed Stream-Buffered-0.03
--> Working on Devel::StackTrace
Fetching http://www.cpan.org/authors/id/D/DR/DROLSKY/Devel-StackTrace-2.04.tar.gz ... OK
Configuring Devel-StackTrace-2.04 ... OK
Building Devel-StackTrace-2.04 ... OK
Successfully installed Devel-StackTrace-2.04
--> Working on Devel::StackTrace::AsHTML
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Devel-StackTrace-AsHTML-0.15.tar.gz ... OK
Configuring Devel-StackTrace-AsHTML-0.15 ... OK
Building Devel-StackTrace-AsHTML-0.15 ... OK
Successfully installed Devel-StackTrace-AsHTML-0.15
--> Working on Apache::LogFormat::Compiler
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/Apache-LogFormat-Compiler-0.36.tar.gz ... OK
Configuring Apache-LogFormat-Compiler-0.36 ... OK
==> Found dependencies: POSIX::strftime::Compiler
--> Working on POSIX::strftime::Compiler
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/POSIX-strftime-Compiler-0.44.tar.gz ... OK
Configuring POSIX-strftime-Compiler-0.44 ... OK
Building POSIX-strftime-Compiler-0.44 ... OK
Successfully installed POSIX-strftime-Compiler-0.44
Building Apache-LogFormat-Compiler-0.36 ... OK
Successfully installed Apache-LogFormat-Compiler-0.36
--> Working on Filesys::Notify::Simple
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/Filesys-Notify-Simple-0.14.tar.gz ... OK
Configuring Filesys-Notify-Simple-0.14 ... OK
Building Filesys-Notify-Simple-0.14 ... OK
Successfully installed Filesys-Notify-Simple-0.14
--> Working on HTTP::Message
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/HTTP-Message-6.44.tar.gz ... OK
Configuring HTTP-Message-6.44 ... OK
==> Found dependencies: IO::HTML, Encode, LWP::MediaTypes, HTTP::Date, Clone
--> Working on IO::HTML
Fetching http://www.cpan.org/authors/id/C/CJ/CJM/IO-HTML-1.004.tar.gz ... OK
Configuring IO-HTML-1.004 ... OK
==> Found dependencies: Encode
--> Working on Encode
Fetching http://www.cpan.org/authors/id/D/DA/DANKOGAI/Encode-3.19.tar.gz ... OK
Configuring Encode-3.19 ... OK
Building Encode-3.19 ... OK
Successfully installed Encode-3.19 (upgraded from 2.97)
Building IO-HTML-1.004 ... OK
Successfully installed IO-HTML-1.004
--> Working on LWP::MediaTypes
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/LWP-MediaTypes-6.04.tar.gz ... OK
Configuring LWP-MediaTypes-6.04 ... OK
Building LWP-MediaTypes-6.04 ... OK
Successfully installed LWP-MediaTypes-6.04
--> Working on HTTP::Date
Fetching http://www.cpan.org/authors/id/O/OA/OALDERS/HTTP-Date-6.06.tar.gz ... OK
Configuring HTTP-Date-6.06 ... OK
==> Found dependencies: Time::Zone
--> Working on Time::Zone
Fetching http://www.cpan.org/authors/id/A/AT/ATOOMIC/TimeDate-2.33.tar.gz ... OK
Configuring TimeDate-2.33 ... OK
Building TimeDate-2.33 ... OK
Successfully installed TimeDate-2.33
Building HTTP-Date-6.06 ... OK
Successfully installed HTTP-Date-6.06
--> Working on Clone
Fetching http://www.cpan.org/authors/id/G/GA/GARU/Clone-0.46.tar.gz ... OK
Configuring Clone-0.46 ... OK
Building Clone-0.46 ... OK
Successfully installed Clone-0.46
Building HTTP-Message-6.44 ... OK
Successfully installed HTTP-Message-6.44
--> Working on HTTP::Entity::Parser
Fetching http://www.cpan.org/authors/id/K/KA/KAZEBURO/HTTP-Entity-Parser-0.25.tar.gz ... OK
Configuring HTTP-Entity-Parser-0.25 ... OK
==> Found dependencies: JSON::MaybeXS, HTTP::MultiPartParser
--> Working on JSON::MaybeXS
Fetching http://www.cpan.org/authors/id/E/ET/ETHER/JSON-MaybeXS-1.004005.tar.gz ... OK
Configuring JSON-MaybeXS-1.004005 ... OK
==> Found dependencies: Cpanel::JSON::XS
--> Working on Cpanel::JSON::XS
Fetching http://www.cpan.org/authors/id/R/RU/RURBAN/Cpanel-JSON-XS-4.37.tar.gz ... OK
Configuring Cpanel-JSON-XS-4.37 ... OK
Building Cpanel-JSON-XS-4.37 ... OK
Successfully installed Cpanel-JSON-XS-4.37
Building JSON-MaybeXS-1.004005 ... OK
Successfully installed JSON-MaybeXS-1.004005
--> Working on HTTP::MultiPartParser
Fetching http://www.cpan.org/authors/id/C/CH/CHANSEN/HTTP-MultiPartParser-0.02.tar.gz ... OK
Configuring HTTP-MultiPartParser-0.02 ... OK
==> Found dependencies: Test::Deep
--> Working on Test::Deep
Fetching http://www.cpan.org/authors/id/R/RJ/RJBS/Test-Deep-1.204.tar.gz ... OK
Configuring Test-Deep-1.204 ... OK
Building Test-Deep-1.204 ... OK
Successfully installed Test-Deep-1.204
Building HTTP-MultiPartParser-0.02 ... OK
Successfully installed HTTP-MultiPartParser-0.02
Building HTTP-Entity-Parser-0.25 ... OK
Successfully installed HTTP-Entity-Parser-0.25
--> Working on HTTP::Headers::Fast
Fetching http://www.cpan.org/authors/id/T/TO/TOKUHIROM/HTTP-Headers-Fast-0.22.tar.gz ... OK
Configuring HTTP-Headers-Fast-0.22 ... OK
Building HTTP-Headers-Fast-0.22 ... OK
Successfully installed HTTP-Headers-Fast-0.22
Building Plack-1.0050 ... OK
Successfully installed Plack-1.0050
--> Working on FCGI::ProcManager
Fetching http://www.cpan.org/authors/id/A/AR/ARODLAND/FCGI-ProcManager-0.28.tar.gz ... OK
Configuring FCGI-ProcManager-0.28 ... OK
Building FCGI-ProcManager-0.28 ... OK
Successfully installed FCGI-ProcManager-0.28
<== Installed dependencies for .. Finishing.
34 distributions installed
---> Patching Plack::Handler::FCGI to work with mod_fcgi ...
~/extlib/lib/perl5 ~
patching file Plack/Handler/FCGI.pm
~
--> 1d8626bf416
STEP 12/12: CMD /usr/libexec/s2i/run
COMMIT ubi8/perl-526:1-testapp
--> 7e7d1854be2
Successfully tagged localhost/ubi8/perl-526:1-testapp
7e7d1854be28e5afad823087b1f2783b840ff22a6ba41919143a55c816a5b163
Waiting for container start... 1

[INFO] Testing the HTTP response for <http://10.88.0.30:8080/>...


[INFO] Testing the HTTP response for <http://10.88.0.30:8080/>...


[INFO] Changing source code of application...


[INFO] Testing the HTTP response for <http://10.88.0.30:8080/>...

Test for image 'ubi8/perl-526:1' FAILED (exit code: 1)
Test for image 'ubi8/perl-526:1' FAILED (exit code: 1)

[INFO] Cleaning up the test application...

c364984684e7981fa583fc1386aa0b869242c1c85dec071ea35d099dc180c5d6
Error: no container with ID or name "c364984684e7981fa583fc1386aa0b869242c1c85dec071ea35d099dc180c5d6" found: no such container
rm: cannot remove '/tmp/tmp.2I4B5Qr4K7/cid': No such file or directory

[INFO] Cleaning up the test application image...

Untagged: localhost/ubi8/perl-526:1-testapp
Deleted: 7e7d1854be28e5afad823087b1f2783b840ff22a6ba41919143a55c816a5b163
Deleted: 1d8626bf416ec7e7d4e7bfd836d6c03494f138e167a5393a7acc96ce29f04dfe
Deleted: 131db2da5eea0702ac159e5527a556a5890c8a9c1e215e4200cf83d30932e7b0
Deleted: 23e464311607fff41a02f305a33608a0cb0eca6fa048346ea0eaf0c97ed3545f
Deleted: 085945e09a0c3294047502a52ebd693e9f521a0ebe694d3ba788e70c42ed11eb
Deleted: 6fcb162653b70e02ba88db279bf7a2df7a335a5ac97d32d2afc98be2dac4c7e5
Deleted: 5401b0710b4d0471edf84e429e335ac4e6c78b53312cc642e3a19d9c18542246
Deleted: f9fb29d5963871498064226b6b143d1da061b9902e7eaeed87e52a9cee84fd18
Deleted: 65845f69a3306c500c5c5eb355be40dc85b5f1d76013d8ac9fa256110b37b6f4
Deleted: 9b9b1f3d693faa952ecb886fae5ba6d019b7af9a7ccb7af5500a7c6168215e5c
Deleted: 0f9368d5228b3439d74b81150b923f53e07dc298f0cda602a3cfea0e5842445d

[INFO] All tests for the psgi-hot_deploy finished successfully....

-----------------------------------------------
Running test test_psgi_variables (starting at 2023-09-11 08:20:46-04:00) ...
-----------------------------------------------

[INFO] Starting tests for psgi-variables....


[INFO] Build the test application image...

hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: 	git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: 	git branch -m <name>
Initialized empty Git repository in /root/s2i-perl-container/test/psgi-variables/.git/
fatal: not in a git directory
fatal: detected dubious ownership in repository at '/root/s2i-perl-container/test/psgi-variables'
To add an exception for this directory, call:

	git config --global --add safe.directory /root/s2i-perl-container/test/psgi-variables
STEP 1/12: FROM ubi8/perl-526:1
STEP 2/12: LABEL "io.openshift.s2i.build.image"="ubi8/perl-526:1"       "io.openshift.s2i.build.source-location"="file:///root/s2i-perl-container/5.26-mod_fcgid/test/psgi-variables"
--> 949ce26ff3a
STEP 3/12: USER root
--> 9a796587a9f
STEP 4/12: COPY upload/src/ /tmp/src
--> 338aaf40dae
STEP 5/12: RUN chown -R 1001:0 /tmp/src
^C
Resources info:
Memory:
              total        used        free      shared  buff/cache   available
Mem:          1.7Gi       205Mi       586Mi        18Mi       984Mi       1.3Gi
Swap:            0B          0B          0B
Storage:
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        870M     0  870M   0% /dev
tmpfs           889M   84K  889M   1% /dev/shm
tmpfs           889M   17M  872M   2% /run
tmpfs           889M     0  889M   0% /sys/fs/cgroup
/dev/vda1        20G  4.6G   16G  23% /
tmpfs           178M     0  178M   0% /run/user/0
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/7d8eedc5acc6701196939d1d5b8f261fa26b50a9fba1111cac1f6a646b20c0f6/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/ba42648e659717e6b1b95fc93fc10e8ff40f10a0f36ae858bc3c408f3d7f02cc/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/1b174bc681101edd76374acdc423474f41513acf2a0588d023d0f50afe208a52/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/cc9d07208e147c46ce0f290037023e85aad83f81e94f99311003aeca3646d279/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/605dfed1ac8b7af89aafe4ada7dc1addb145746a22d70d401f09eab2150748f9/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/311f1da86cddaebe79f2e5ca559f2e830d9eba582fb5019a46f44d258755250d/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/27138d9ba981ee66df379b1dfff80785f378870e0c3357b1cb51ef2fe3e9e6d4/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/d006d26d90dfcdc9f4d726f4b4a3cca5ad7fa819f7314e52170ffb3c136d864f/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/ea85bb052b3768f06cc1b9c704865e4cb78e39a5cdd3f36381527e5e3ded71e2/merged
overlay          20G  4.6G   16G  23% /var/lib/containers/storage/overlay/f91f056f7e89828fe6aab5d8de6e3f46aa9e4c5566cff7c8335a7fb9469e295a/merged
CPU
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              1
On-line CPU(s) list: 0
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Red Hat
CPU family:          6
Model:               134
Model name:          Intel Xeon Processor (Icelake)
BIOS Model name:     RHEL 7.6.0 PC (i440FX + PIIX, 1996)
Stepping:            0
CPU MHz:             2294.608
BogoMIPS:            4589.21
Virtualization:      VT-x
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            4096K
L3 cache:            16384K
NUMA node0 CPU(s):   0
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves wbnoinvd arat avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid fsrm md_clear arch_capabilities
Tests were run for image ubi8/perl-526:1
Uncompressed size of the image: 956MB
Compressed size of the image: 444MB

===============================================
Test cases results:

 [PASSED] for 'appnamenotset' test_sample_test_app (00:00:17)
 [PASSED] for 'appnamenotset' test_binpath (00:00:11)
 [PASSED] for 'appnamenotset' test_psgi (00:00:59)
 [PASSED] for 'appnamenotset' test_psgi_hot_deploy_without_reload (00:00:55)
 [FAILED] for 'appnamenotset' test_psgi_hot_deploy_with_reload (00:00:56)

Tests for ubi8/perl-526:1 failed.
Examining image ID files in $APP_ID_FILE_DIR=/tmp/tmp.xI4gADetSF
Examining CID files in $CID_FILE_DIR=/tmp/tmp.jt3BmQ35Rg

@michal-josef-spacek
Copy link
Member

@phracek Interesting. Are you using podman or docker?

@michal-josef-spacek
Copy link
Member

ok, I could test on RHEL8 host. Previous one was on Fedora 38

@michal-josef-spacek
Copy link
Member

@phracek
I tested on RHEL 8.9 (the actual last version) with the last s2i-perl-container git commit.
There is no issue.
In this case, I am using podman. Output is a bit different from that on Fedora, but functional.

btw: I am curious about your:

[INFO] Cleaning up the test application...

c364984684e7981fa583fc1386aa0b869242c1c85dec071ea35d099dc180c5d6
Error: no container with ID or name "c364984684e7981fa583fc1386aa0b869242c1c85dec071ea35d099dc180c5d6" found: no such container
rm: cannot remove '/tmp/tmp.2I4B5Qr4K7/cid': No such file or directory

As I remember, this was an issue some time ago (probably docker rm issue), not at this time.
What version do you test?

@zmiklank
Copy link
Contributor

As I remember, this was an issue some time ago (probably docker rm issue), not at this time. What version do you test?

Indeed, AFAIK this issue was fixed with this PR.

@michal-josef-spacek
Copy link
Member

As I remember, this was an issue some time ago (probably docker rm issue), not at this time. What version do you test?

Indeed, AFAIK this issue was fixed with this PR.

@zmiklank Yes, but this is not the root cause of this issue. This is only the context of reported results.

In the case of this issue, I don't know. I haven't enough information.

@zmiklank
Copy link
Contributor

But weird is that the second log is from 2023-09-11 when the "trailing --rm" bug should have been fixed already.

@zmiklank
Copy link
Contributor

@phracek could you please confirm that you are testing on latest master?

@phracek
Copy link
Member Author

phracek commented Sep 13, 2023

@phracek could you please confirm that you are testing on latest master?

@zmiklank As far as I know we get the sources by this commands:

        dnf install -y git ansible
        git clone <this_repo> /root/s2i-perl-container
        cd /root/s2i-perl-container
        git fetch origin +refs/pull/*:refs/remotes/origin/pr/*
        git checkout origin/pr/$PR_NUMBER/head
        git submodule update --init

or I don't follow your question.

@michal-josef-spacek
Copy link
Member

No reproducer. No any log with error. I am closing the issue.

Tested on Fedora 38 and RHEL 8.9 in last commit and everything is working.

@michal-josef-spacek
Copy link
Member

@phracek said that the issue is on RHEL 8.8

I tested on RHEL 8.8 in container with command:

make test TARGET=rhel8 VERSIONS=5.26-mod_fcgid

Result is pretty fine

 [PASSED] for 'appnamenotset' test_sample_test_app (00:00:21)
 [PASSED] for 'appnamenotset' test_binpath (00:00:12)
 [PASSED] for 'appnamenotset' test_psgi (00:01:36)
 [PASSED] for 'appnamenotset' test_psgi_hot_deploy_without_reload (00:01:22)
 [PASSED] for 'appnamenotset' test_psgi_hot_deploy_with_reload (00:01:18)
 [PASSED] for 'appnamenotset' test_psgi_variables (00:01:35)
 [PASSED] for 'appnamenotset' test_warningonstderr (00:00:10)
 [PASSED] for 'appnamenotset' test_fcgi (00:00:12)
 [PASSED] for 'appnamenotset' test_npm (00:00:14)
 [PASSED] for 'appnamenotset' test_from_dockerfile (00:05:05)

@zmiklank
Copy link
Contributor

I am sorry @phracek, but without a valid (minimal) reproducer we are not able to continue debugging this issue. Could you please provide a step-by-step sequence for reproducing this issue?

@zmiklank
Copy link
Contributor

Maybe the problem is in some kind of specific setup?

@michal-josef-spacek
Copy link
Member

My log from build and test:
mspacek-rhel8-test-s2i-perl-container.log.gz

@michal-josef-spacek
Copy link
Member

As I understand, my repositories are:

Red Hat Universal Base Image 8 (RPMs) - BaseOS  841 kB/s | 716 kB     00:00
Red Hat Universal Base Image 8 (RPMs) - AppStre 2.5 MB/s | 2.9 MB     00:01
Red Hat Universal Base Image 8 (RPMs) - CodeRea 170 kB/s | 100 kB     00:00

From issue:

Red Hat Enterprise Linux 8 for x86_64 - BaseOS  130 MB/s |  65 MB     00:00
Red Hat Enterprise Linux 8 for x86_64 - AppStre 120 MB/s |  59 MB     00:00
Red Hat Universal Base Image 8 (RPMs) - BaseOS  2.8 MB/s | 716 kB     00:00
Red Hat Universal Base Image 8 (RPMs) - AppStre 9.7 MB/s | 2.9 MB     00:00
Red Hat Universal Base Image 8 (RPMs) - CodeRea 499 kB/s |  99 kB     00:00

@michal-josef-spacek
Copy link
Member

The issue is still here. See nightly builds.

@phracek
Copy link
Member Author

phracek commented Apr 4, 2024

It should be fixed by #298

@phracek phracek closed this as completed Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants