Upgrading to v2, error *4848 port is already used* in zulu

[hantsy]

Platform:

• Ubuntu
• macOS
• Windows

Runner type:

• Hosted
• Self-hosted

Follow the migrate guide, after upgrading to v2, all my ci using Glassfish failed due to a 4848 port is already used,

https://github.com/hantsy/jakartaee9-starter-boilerplate/blob/b3e51fcd857768c05e6104ea940f0ecd283c4698/.github/workflows/it-with-arq-glassfish-managed.yml#L22

but switch back to v1, work again.

https://github.com/hantsy/jakartaee9-starter-boilerplate/blob/master/.github/workflows/it-with-arq-glassfish-managed.yml#L22

[dmitry-shibanov]

Hello @hantsy. I think it's not related to setup-java task, because I forked your repository and could reproduce the issue with setup-java@v1.

• it-with-arq-glassfish-managed
• it-with-arq-glassfish-remote

[hantsy]

The tests in my master branch are all passed now.

You can check the commit log, I was trying to upgrade to v2, failed. Then I have to return back to v1, worked again.

[hantsy]

The same problem when using v2 with Payara(a fork of Glassfish), check the details.
https://github.com/hantsy/cargotracker/runs/2311107552#step:6:1543

[hantsy]

Not sure the workflows in the repository cached the old JDK, when returning to the v1, some former successful workflows in https://github.com/hantsy/cargotracker failed now.

[dmitry-shibanov]

Hello @hantsy. We investigated the issue and don't think it's something task related. As we can see the issue reproduces on both v1 and v2 versions:

• v2: v2 passed and v2 failed
• v1: v1 failed

[hantsy]

So strange, also reported to Payara, also do not know the reason.

And the most strange is the GlassFish v6.0/Jakarta EE 9 testing codes failed now.

[hantsy]

Till now, almost all Payara/GlassFish related workflows which worked well in the past months or years are failed in all of my repositories.

• https://github.com/hantsy/jakartaee8-starter-boilerplate
• https://github.com/hantsy/jakartaee9-starter-boilerplate
• https://github.com/hantsy/cargotracker/

I was afraid the port conflict is caused by the workflow execution in parallel. In https://github.com/hantsy/cargotracker/, I removed other Payara-related workflows, only left Payara managed adapter(the only one that used 4848 port) to run the testing codes, failed. No doubt the testing codes using the same Maven profiles passed on my local system.

I tried to add a script to print which process occupied 4848. https://github.com/hantsy/cargotracker/blob/master/.github/workflows/it-with-arq-payara-managed.yml#L54, but it seems this command did not work on Github Actions.

And in the above list, the projects also configured multi workflows for WildFly, OpenLiberty, these workflows never failed.

[maxim-lobanov]

@hantsy , every workflow is run on the separate clean VM that is isolated from other VMs so any parallel conflicts are eliminated.
As for the port 4848, the command works as expected. But before the steps Run Glassfish Server and Run integration test with -Parq-payara-managed port is free. So looks like something goes wrong during this step.

[hantsy]

every workflow is run on the separate clean VM that is isolated from other VMs so any parallel conflicts are eliminated.

It is great to confirm this.

You can check the commit log, before 5, April (setup-java v2 pr created at that moment), the workflow for glassfish-remote is like the following which worked well for months.

https://github.com/hantsy/jakartaee9-starter-boilerplate/blob/6c5a20a8335cc9cc94e763d1020ac0e4668b034b/.github/workflows/it-with-arq-glassfish-remote.yml

In the newest version of https://github.com/hantsy/jakartaee9-starter-boilerplate, the glassfish-remote failed at Run Glassfish server task, https://github.com/hantsy/jakartaee9-starter-boilerplate/runs/2367936475, it means before starting Glassfish in my scripts, 4848 is already in use.

I am not sure why glassfish-managed worked again in the newest version in https://github.com/hantsy/jakartaee9-starter-boilerplate, it failed most of the time in the past 2 weeks(after 5, April). The managed here means the lifecycle of running Glassfish is controlled by the tests framework(Arquillian) itself.

[hantsy]

Created a new Circle CI config, https://github.com/hantsy/cargotracker/blob/master/.circleci/config.yml#L29, the build is successful. The scripts are 100% same as the Github actions payara-managed workflow, https://github.com/hantsy/cargotracker/blob/master/.github/workflows/it-with-arq-payara-managed.yml#L74

The Circle CI environment never reports 4848 is already in use error.

And this error never occurred on my local system. Please note the workflow files I listed above #158 (comment) are not newly added, they have run on Github actions for months or years but never encountered the problem till in these two weeks at the moment setup-java v2 was released.

I also tried to print the port info in some steps, https://github.com/hantsy/cargotracker/blob/master/.github/workflows/it-with-arq-payara-managed.yml#L54, but Github actions will exit quickly when executing this scripts. So I do not know which program was using 4848 at all.

[maxim-lobanov]

I suggest trying add the following step to the failed workflow right before invocation of your tool:

- run: sudo netstat -lpn

It should list all busy ports.

I can't reproduce the issue right after build starts, port is free:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:8084            0.0.0.0:*               LISTEN      753/mono            
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      547/systemd-resolve 
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      718/sshd: /usr/sbin 
tcp        0      0 127.0.0.1:41399         0.0.0.0:*               LISTEN      715/containerd      
tcp6       0      0 :::22                   :::*                    LISTEN      718/sshd: /usr/sbin 
udp        0      0 127.0.0.53:53           0.0.0.0:*                           547/systemd-resolve 
udp        0      0 10.1.0.4:68             0.0.0.0:*                           527/systemd-network 
udp        0      0 127.0.0.1:323           0.0.0.0:*                           676/chronyd         
udp6       0      0 ::1:323                 :::*                                676/chronyd         
raw6       0      0 :::58                   :::*                    7           527/systemd-network

[hantsy]

I am crazy, really not sure why it happened.

[hantsy]

@maxim-lobanov In these two days, there are some PRs merged into the repos, it seems everything becomes as before. The broken workflows(due to 4848 port issues) run successfully.

• https://github.com/hantsy/jakartaee8-starter-boilerplate
• https://github.com/hantsy/jakartaee9-starter-boilerplate
• https://github.com/hantsy/cargotracker/

[maxim-lobanov]

That sounds really weird because neither actions/setup-java nor Hosted runners (both successful and failed builds are run on 20210412.1 image version) were updated. I wonder if it could it be an issue with some kind of dependencies

[hantsy]

@maxim-lobanov maybe a system package that JDK is dependent on caused this issue. Thanks for your patience. I close this issue.

[hantsy]

@maxim-lobanov This issue occurred again, https://github.com/hantsy/jakartaee9-starter-boilerplate/runs/4853696050?check_suite_focus=true#step:4:2266