Executing jar without system java (JRE) on linux

[Wunarg]

This issue is about a public attempt of transitioning my game on linux to have a bundled java version since systems java is no longer accessable.
Its public to possibly help people at steam to provide better assistance, it may also be usefull as an example. This is mostly due to @TTimo offering/asking about this.

Previous method:
There was a .sh (bash) file
-this bash file checked the systems java version via:
version=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}' | awk -F '.' '{sub("^$", "0", $2); print $1$2}')
-this translated the version into a 2 digit number to then compare it and check if the java version is high enough
-the jar file is then executed via:
java -jar Cube\ Chaos.jar "$@" (the "$@" apparently
-this is a simple jar file produced by a Java IDE (in my case Eclipse) which is the game itself

This is no longer functional because steam launches the bash file without access to the systems java.

Disclaimer:
I have close to no experience with bash scripts, most of this is snippeted together.
Also I am releasing an update on the first of the next month so I may be a little slow to respond or do work on this.
I can only test on linux very slowly due to the virual machine beeing very slow and finicky, as such I have asked one of the people of the community for help, though this is obviouly still messy.

Attempt:
-This whole attempt is on a seperate devtest branch to make sure normal use is not impacted
-I have downloaded the newest linux openjdk jdk (oracle is not free to distribute) and added it to the folder which is the package
-I have modified the bash to be:

#!/bin/bash

cd ./linuxJdk/bin

java -jar ../../Cube\ Chaos.jar

read

-the read command is just there to keep the terminal open for the purpose of testing
-I have attempted it without the cd going into the java bin subfolder but without that the java command does not work, even if changed to /linuxJdk/bin/java -jar Cube\ Chaos.jar to include the path there.
-This does launch the file in some way however this approach has 2 problems so far:
--Presumably due to beeing opened from a different path the game cannot find its own assets, I have no idea how to possibly be in the java path as well as in the main game path, but likely am just missing some bash knowledge. Could also be some other problem who knows.
--For some reason when the bash is executed manually steam automaticly tries to launch as well and adds its log to the bash terminal, this is quite perplexing considering the previous bash file did not do this. I am currently using https://github.com/code-disaster/steamworks4j to connect to steam with the game for stuff like friends list info and achievements, however this shouldent launch steam nor has it in the past. Additionally the game is made to work fine even without steam. This is not strictly required to fix but would make the game more susceptible to future problems.

Questions:
-Is there a common way to launch jar games via steam which doesnt require this messy bash workaround?
-Any ideas on how to resolve the bash folder issue?
-Any idea why manually executing the bash also launches steam?

[smcv]

the read command is just there to keep the terminal open for the purpose of testing

There is a way to do this without having to modify your game's startup script for everyone. If you set your game's Launch Options in the Steam user interface (for just the user you are using to test it) to

PRESSURE_VESSEL_SHELL=instead %command%

then when you launch the game, you will get an interactive shell in an xterm terminal. That shell is inside the Steam Linux Runtime container, and you can experiment with running things interactively. If you type "$@" at the shell prompt (including the double quotes!), it will run the same command that Steam would normally have run when it launched your game

[smcv]

I can only test on linux very slowly due to the virual machine beeing very slow and finicky

If you have a spare computer that is capable of running Steam and your game (perhaps an old one that you no longer use because your main computer is better), you might find it easier to use that as a test environment. As an example, my test setup includes a laptop from 2012 (a Lenovo Thinkpad T520 that one of my colleagues left for recycling), which works fine with various versions of Debian or Ubuntu.

Obviously I mostly use something better, but the T520 is useful when I'm trying to improve performance, because it's so slow that it makes any performance loss/gain more obvious!

[smcv]

Any ideas on how to resolve the bash folder issue?

If your game expects the current working directory to be its own top level directory, then something like this would perhaps work better, without the cd (untested):

#!/bin/bash
set -e
exec ./linuxJdk/bin/java -jar 'Cube Chaos.jar' "$@"

set -e makes the script immediately fail if a command fails, unless you specifically ignore the failure, which makes shell scripts behave a bit more like every other programming language. 99% of the time this is what you want.

The exec makes the script replace itself with the JRE instead of continuing to run, which eliminates an unnecessary shell process. You can leave that out if you don't want that behaviour.

The "$@" lets you pass command-line arguments to your game, which otherwise would be eaten by the shell script.

I personally find it easier to read scripts if I put quotes around filenames that contain spaces, instead of using backslash escapes, but that's a matter of personal taste rather than a technical requirement.

If the JRE can't find its own required files when you do this, you might need to set some environment variables or something, for example perhaps something like this (this is a guess, I've forgotten almost everything that I once knew about Java; also untested):

#!/bin/bash
set -e
export JAVA_HOME="$(pwd)/linuxJdk"
export PATH="$JAVA_HOME/bin:$PATH"
exec java -jar 'Cube Chaos.jar' "$@"

Or perhaps setting CLASSPATH might also be necessary (it's a colon-separated list of locations to look for Java files, but I don't know what assumptions the JRE makes about it).

[smcv]

For some reason when the bash is executed manually steam automaticly tries to launch as well and adds its log to the bash terminal, this is quite perplexing considering the previous bash file did not do this. I am currently using https://github.com/code-disaster/steamworks4j to connect to steam with the game for stuff like friends list info and achievements, however this shouldent launch steam nor has it in the past.

I don't know what's going on here. Are you sure that you're "comparing apples with apples" and keeping all other relevant factors the same, like for instance using the same version of steamworks4j?

If I understand Valve's policies correctly, the only configuration that really needs to work for a Steam game is launching it via Steam. If you want to provide a version for "generic Linux" that doesn't require Steam, you can do that too, but Valve isn't going to require you to do that.

If you do provide a non-Steam version of your game, it doesn't necessarily need to be identical to the version you ship on Steam - for example you could include a bundled copy of the OpenJDK JRE and steamworks4j in your Steam game, but you could leave those out and require the user to provide their own JRE in the non-Steam version.

It would also be OK for your game's startup script to detect whether it was run by Steam or directly, and do different things in each case, perhaps something like this (untested!):

#!/bin/bash
set -e

if [ -n "${SteamAppId-}" ]; then
    echo "running under Steam, app ID ${SteamAppId}"
    export JAVA_HOME="$(pwd)/linuxJdk"
    export PATH="$JAVA_HOME/bin:$PATH"
else
    echo "not running under Steam"
fi

exec java -jar 'Cube Chaos.jar' "$@"

(@TTimo: is there a better/more canonical way to detect whether we're running under Steam than testing whether $SteamAppId is set?)

[smcv]

I have attempted it without the cd going into the java bin subfolder but without that the java command does not work

Please say more than "does not work": the only information I have is what you give me. If there are error messages, please copy/paste the error message.

even if changed to /linuxJdk/bin/java -jar Cube\ Chaos.jar to include the path there

/linuxJdk/bin/java is an absolute path (it's telling the shell to look for linuxJdk at the top level of the filesystem), like saying C:\linuxJdk\bin\java in Windows. If what you mean is "look for linuxJdk in the current working directory" then the way to spell that is ./linuxJdk/bin/java or $(pwd)/linuxJdk/bin/java.

[smcv]

Is there a common way to launch jar games via steam which doesnt require this messy bash workaround?

The way Steam launch commands work means that it needs something that you could run by just typing its absolute path, so, no, you need either a shell script wrapper like the one you've written, or a C/C++ executable.

In my experience, most native Linux games on Steam start from a shell script wrapper - but it can often be something very short and simple, like the examples in #741 (comment).

[smcv]

Is this your game? https://steamdb.info/app/1958340/info/

I notice that it has been tested on Steam Deck, and is flagged as "Playable", "Recommended runtime: Proton stable" in https://steamdb.info/app/1958340/info/.

If that test result is accurate, then one option if you cannot easily test on Linux would be to only provide the Windows version, and rely on Proton to make your game available to your Linux players. A well-tested Windows version, combined with Proton, might actually give you better compatibility than a native Linux port that hasn't been tested much.

But, you're not the only developer who is trying to publish Java games with a native Linux variant, so it's probably worthwhile to invest a bit of time in seeing whether we can make this work. If you don't mind sending a few Steam keys to smcv at collabora dot com (and also a suitable beta branch password so we can access the test branch where you've included a bundled JRE) then my colleagues and I can use your game as our sample Java game, and see whether we can provide a better recommendation.

[Wunarg]

PRESSURE_VESSEL_SHELL=instead %command%
Good to know

#!/bin/bash
set -e
exec ./linuxJdk/bin/java -jar 'Cube Chaos.jar' "$@"

Tried this on windows (to see if it could find the file and work so far) resulting in the following:

Tested the other one:

#!/bin/bash
set -e
export JAVA_HOME="$(pwd)/linuxJdk"
export PATH="$JAVA_HOME/bin:$PATH"
exec java -jar 'Cube Chaos.jar' "$@"

This works on linux with no immediatly visible downsides or sideeffects, very good. I will push this to the beta branch soon (if possible) and if nothing bad happens to main branch some time later. Thank you!

I don't know what's going on here. Are you sure that you're "comparing apples with apples" and keeping all other relevant factors the same, like for instance using the same version of steamworks4j?

Everything else was the same except the new linuxjdk folder and the Linux.sh bash file, just very strange but I guess not really something I need to worry about just thought it worth mentioning. (Also don't need to detect if steam is running that part works fine with the actually game jar)

Please say more than "does not work": the only information I have is what you give me. If there are error messages, please copy/paste the error message.

Yeah sorry, I think it just straight up couldent find the file, probably messing up the path + needing the actuall java path set.

Is this your game? https://steamdb.info/app/1958340/info/

Correct

I notice that it has been tested on Steam Deck, and is flagged as "Playable", "Recommended runtime: Proton stable" in https://steamdb.info/app/1958340/info/.

Ye, sorry forgot to repeat this part when moving to this new issue from the old one. Proton was reported to be slow and laggy with my game, so I want to support native linux when possible, especially since java handles most of the compatability stuff.

But, you're not the only developer who is trying to publish Java games with a native Linux variant, so it's probably worthwhile to invest a bit of time in seeing whether we can make this work. If you don't mind sending a few Steam keys to smcv at collabora dot com (and also a suitable beta branch password so we can access the test branch where you've included a bundled JRE) then my colleagues and I can use your game as our sample Java game, and see whether we can provide a better recommendation.

Who are you? I dont understand how you don't have access to the game naturally as steam personel, but also need access for your collagues to use as a sample and for better recommendation?

Thank you for your assistance and info so far!

[smcv]

Tried this on windows (to see if it could find the file and work so far) resulting in the following: ![Image](https://github.com/user-attachments/assets/1f3ba823-f75b-4362-b9ec-2fa30803ca3c)

If possible please copy/paste text into code blocks in your comments, rather than taking screenshots of it - that way, we can
quote from it in replies. (Also, all the usual reasons about accessibility tools, etc.)

Transcribing some of it by hand (please excuse my abbreviations and typos):

bash: /d/Game Dev/…/linuxJdk/bin/java: cannot execute binary file: Exec format error

"Exec format error" means "I tried to run this but it's for another platform".

In this case, if you're in some sort of developer shell on Windows (perhaps "git bash" or similar?), it's normal and expected that you can't run linuxJdk/bin/java, because linuxJdk/bin/java is (presumably) a Linux ELF executable, and Windows only knows how to run Windows PE executables.

I see that your game's depot contains a jre/bin/java.exe, which is presumably part of a bundled Windows JRE. If you tried to run that on Linux, you'd expect it to fail (unless something like Wine/Proton is translating it). Not being able to run linuxJdk/bin/java on Windows is exactly the same, but the other way around.

The Permission denied error might be part of the same problem, or it might indicate that you need to chmod +x the executables in linuxJdk/bin/ so that Linux systems will be able to run them.

[smcv]

As a side note, I see that your game has one big depot, containing both Windows and Linux files. I believe the typical way to structure this would be to have one depot for data files that are common to platforms (containing Cube Chaos.jar, GameData and GeneralData), a second depot marked as Windows-only for Windows-specific things (Cube Chaos.exe and jre/), and a third depot marked as Linux-only for Linux-specific things (linuxJdk/ and Linux.sh). That way, your Windows players won't be required to download all the Linux files, and your Linux players won't be required to download all the Windows files.

The open-source game Endless Sky might be a useful real-world example of this: their Endless Sky Data has their equivalent of your GameData and so on, their Endless Sky Win64 is their equivalent of your Cube Chaos.exe and jre/, and their Endless Sky Linux64 is their equivalent of your linuxJdk/ and Linux.sh).

[smcv]

Tested the other one [that sets JAVA_HOME and PATH]. This works on linux with no immediatly visible downsides or sideeffects, very good. I will push this to the beta branch soon (if possible) and if nothing bad happens to main branch some time later. Thank you!

The one warning I would give you about this version is that if the setup is not quite right, but you run it on a system that happens to have a system JRE in /usr/bin, it might work "accidentally" by falling back to the system JRE - but then it still wouldn't necessarily work on other systems.

You didn't say whether you'd tested this under Steam Linux Runtime 1.0, under Legacy runtime 1.0, or outside Steam. For the most thorough compatibility test, I'd recommend testing it under Steam Linux Runtime 1.0, or on a system that doesn't have a JRE installed system-wide at all (for example a Steam Deck), or both.

[smcv]

Who are you? I dont understand how you don't have access to the game naturally as steam personel, but also need access for your collagues to use as a sample and for better recommendation?

That's a fair question, and if you're not comfortable with giving out keys for testing, I completely understand that.

I work for a consultancy called Collabora which is helping Valve with their Linux products like the Steam Deck and the Steam Linux Runtime. We don't have the same level of access to Steam infrastructure as the people who work on the infra itself, and the Steam accounts we use for testing are not special: we only have access to games that we've either bought or received a key for.

I wrote quite a lot of the code that runs the Steam Linux Runtime container, which I spoke about at FOSDEM 2020 (information in that presentation is a few years out of date now!) and mentioned in our press release when the Steam Deck shipped.

[Wunarg]

In this case, if you're in some sort of developer shell on Windows (perhaps "git bash" or similar?), it's normal and expected that you can't run linuxJdk/bin/java, because linuxJdk/bin/java is (presumably) a Linux ELF executable, and Windows only knows how to run Windows PE executables.

Hmm that probably makes sense, but then I don't understand how the other linux.sh version works fine on windows even though it runs the same linuxJdk/bin/java. Its not really important but just thought it might be interesting to you.

As a side note, I see that your game has one big depot, containing both Windows and Linux files. I believe the typical way to structure this would be to have one depot for data files that are common to platforms (containing Cube Chaos.jar, GameData and GeneralData), a second depot marked as Windows-only for Windows-specific things (Cube Chaos.exe and jre/), and a third depot marked as Linux-only for Linux-specific things (linuxJdk/ and Linux.sh). That way, your Windows players won't be required to download all the Linux files, and your Linux players won't be required to download all the Windows files.

Ye this seems like a good idea, it wasn't really necessary before since it was just simpler to use 1 depo and with the much smaller size the upload timer and download times werent considerable. But with the larger one uploading it as one depo via zip file takes a bit and sometimes fails, so I have implemented linux/windows depos and am testing them on my beta branch.

I have now sent you an email with 2 steam keys for the game, hope they help.

[smcv]

I have now sent you an email with 2 steam keys for the game, hope they help.

Received, thanks. I'll try to look at this again when time permits.