Make Linux and OSX binaries executable on export

[voidplayer]

---

Bugsquad note: This issue has been confirmed several times already. No need to confirm it further.

---

file doesnt get proper +x permission on export

[akien-mga]

I can confirm this is still the case on the current bd736e5.

[Cheeseness]

This happens even if the template binary has +x

Adding an export_project() function to platform/x11/export/export.cpp that contained a OS::execute() call to run chmod would probably be the way to go, but I'm not familiar enough with Godot's codebase yet to help here.

[akien-mga]

Adding the chmod +x call itself is relatively trivial:

diff --git a/tools/editor/editor_import_export.cpp b/tools/editor/editor_import_export.cpp
index bdf1726..712bc72 100644
--- a/tools/editor/editor_import_export.cpp
+++ b/tools/editor/editor_import_export.cpp
@@ -1372,6 +1372,11 @@ Error EditorExportPlatformPC::export_project(const String& p_path, bool p_debug,
                }
        }

+       List<String> args;
+       args.push_back("+x");
+       args.push_back(p_path);
+       OS::get_singleton()->execute("/usr/bin/chmod",args,false);
+
        String dstfile = p_path.replace_first("res://","").replace("\\","/");
        if (export_mode!=EXPORT_EXE) {

There are several problems though:

• the EditorExportPlatformPC exporter is used for both Windows and Linux templates, and we don't want to chmod +x the Windows binaries
• obviously, we can't use chmod when exporting Linux binaries on a Windows host
• all in all, the gain is little because the executable permission is set for the current system, so if you send the binary to another Linux user, they would receive it with permission 644. The workaround is to put the binary in an archive, which preserves the execution bit.

All in all, I'd be for just making a nice docs page explaining that Linux binaries have to be made executable for distribution.

[nunodonato]

unrelated but related, does anyone know if the exported mac binary also has the same problem? I guess the file inside the .app package needs to have the executable bit set too, right? (its still unix)

[Cheeseness]

the executable permission is set for the current system

Just in case this hasn't been communicated clearly, Godot on Linux doesn't preserve +x on Linux binaries during export even if the template binaries have +x.

[akien-mga]

unrelated but related, does anyone know if the exported mac binary also has the same problem?

The binary is made executable before it's packaged in the .app archive (which is just a .zip, and should preserve permissions). There was an unrelated issues though that @punto- recently fixed: 6962fd9

Just in case this hasn't been communicated clearly, Godot on Linux doesn't preserve +x on Linux binaries during export even if the template binaries have +x.

Because, as I said, the execution permission is registered in your own system and not in the file itself. As the binary is read and written again in another location, these permissions are lost and would have to be manually reapplied.

I'm not sure what's the reason for rewriting the full binary instead of just copying it though.

[Cheeseness]

The binary is made executable before it's packaged in the .app archive (which is just a .zip, and should preserve permissions).

For clarity, .app packages are a folder structure rather than an archive. Mac users still need to extract the .zip before they can launch the .app. You're right that it's in a .zip to preserve the permissions (although as far as I'm aware, that's a result of various unzip implementations respecting this stuff rather than something that's identifiably a part of the zip format).

The same strategy could be viable for making it easier for Windows based developers to ship Godot games on Linux.

Because, as I said, the execution permission is registered in your own system and not in the file itself. As the binary is read and written again in another location, these permissions are lost and would have to be manually reapplied.

Correct. Your comments about sending it to another Linux user seemed to be implying that the exported binary on the same system (computer) would retain +x permissions, but I see now that you were meaning that it's stored at the file system level. Apologies for misunderstanding.

I'm not sure what's the reason for rewriting the full binary instead of just copying it though.

I'm not certain either. Could it be that the export code for making a single binary (containing the game's data) was written first and reused for the pack modes that don't modify the binary?

[akien-mga]

Looks like on Linux we could use chmod directly from <sys/stat.h> directly: http://stackoverflow.com/a/12694263
I guess OSX might have chmod too, though this needs checking.

For Windows, we'll need to find another API, but I'm pretty sure something exists. Any idea about it @vnen?

[punto-]

For osx, the binaries are read from the template and used to create a .zip file aren't they? We don't actually generate the final .app structure on the filesystem and then zip it, we create the .zip programatically. The solution there is to turn on the executable bit for the file inside the zip. That should be portable, we could do it from any OS (if such a bit even exists)

[akien-mga]

The solution there is to turn on the executable bit for the file inside the zip. That should be portable, we could do it from any OS (if such a bit even exists)

That's the idea, but we need to find the APIs that allow to do this for all platforms/executable types.

[punto-]

But the point is this an api in the zip library, it shouldn't be platform
specific.. It's up to the unzip program in the target OS to actually make
it executable when it gets unzipped, if the filesystem supports it.

On 14 July 2016 at 04:33, Rémi Verschelde notifications@github.com wrote:

The solution there is to turn on the executable bit for the file inside
the zip. That should be portable, we could do it from any OS (if such a bit
even exists)

That's the idea, but we need to find the APIs that allow to do this for
all platforms/executable types.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#527 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AGVmPSVQ9-BE6VPVmK_4RytZiiZPdACrks5qVeZagaJpZM4CFTzr
.

[akien-mga]

Well, that's worth looking into for OSX if we can set the executable bit when zipping, but for Linux binaries it won't work as we don't zip them on export.

[Cheeseness]

That's what Godot is already doing for OSX. I think @punto- is suggesting that Linux exports be deployed to zip via a similar method as well.

[akien-mga]

For OSX/zip, we can maybe look at the file attributes in https://github.com/godotengine/godot/blob/master/core/io/zip.h#L100-L108

See e.g. http://unix.stackexchange.com/questions/14705/the-zip-formats-external-file-attribute

That still wouldn't solve the issue of setting the +x when copying the Linux templates (we seem to read the Linux binaries and rewrite it to the chosen destination, thus losing the existing executable bit), but it would be great already to have working OSX .app templates.

[akien-mga]

@Cheeseness Well it doesn't seem to be done so far for OSX, see #4618 (comment)
I don't think that we should start zipping Linux binaries just for that either.

[Cheeseness]

#4618 (comment) isn't in line with what I recall experiencing when working on the two Godot projects I've shipped for Mac OS. I'll double check that next week when I have some more time.

IMO, offering a zipped Linux binary export would be valuable for Windows based developers who want to ship Linux builds.

[akien-mga]

Some more reading about external file attributes in the ZIP API (it's crazy how undocumented this stuff can be o_O): http://stackoverflow.com/a/6297838

My understanding is that we could add something like that after: https://github.com/godotengine/godot/blob/master/platform/osx/export/export.cpp#L401

if (file == pkg_name+".app/Contents/MacOS/"+binary_to_use) {
    fi.external_fa |= 0755 << 16L # -rwxr-xr-x
}

Which would set the UNIX permissions that we want on the binary without (a priori) messing with the DOS permissions.

[vnen]

For what I can gather, the ZIP approach is the only way to provide Linux binaries on Windows with the proper execute bit. If one extract this file on Windows, the NTFS will likely trash Unix permission bits. IME moving files from Windows to Linux always made them 777.

[Cheeseness]

IME moving files from Windows to Linux always made them 777.IME moving files from Windows to Linux always made them 777.

What method are you using to transfer them? This has never ever happened for me.

The only thing that has that kind of behaviour that I know of is Steam, and that's the client setting 777 on all files it downloads (if you listen quietly, you can hear the sound of a million security conscious people facepalming every time a download finishes) rather than anything inherent to the file transfer protocol it uses.

[akien-mga]

I'll give a try at the OSX part this week-end using what I've documented above. If it works, I'll see what can be done for Linux.

[bruvzg]

Have you opened an upstream bug report with the minizip developers?

minizip can set permissions, but unless "made by" field is populated with the specific version with Unix permission support, "unzip"s ignore all permissions. See #33447 > export.cpp > L729-746.

At least on macOS, executable extracted form the ZIP created in this way have correct permissions set. I have tested creating ZIPs on macOS and Windows. And extracting it on macOS using Archiver (system default extractor), Keka and Midnight Commander.

[akien-mga]

@bruvzg Nice catch! So #33447 solves the OSX part of this issue I think, we'd just need to add ZIP export for Linux with similar logic I suppose?

[bryantfhayes]

Is this issue resolved in 3.2.1? Or is it targeting 4.0?

[Calinou]

@bryantfhayes It's not resolved as of 3.2.1. It's targeting 4.0 at least, although a fix could be backported into 3.2.x eventually.

[pouleyKetchoupp]

I've been investigating this issue when exporting as zip from Windows for OSX and got some intriguing results. Depending on which utility is used on Mac to uncompress the zip file, it works in some cases.

On my custom branch based on 3.1:
Using unzip: binary not set as an executable
Using Archive Utility: permissions are set correctly and the game launches

After merging #33447:
It's the other way around.
Using unzip: permissions are set correctly and the game launches
Using Archive Utility: binary not set as an executable

I'm going to try and investigate the differences in how flags are set when zipping files before and after this PR to see if there's a way to make it work in all cases, but in the meantime if someone has an idea about what could be happening I would be interested in any hint.

[TaraSophieDev]

I have the same problem.

[ACanOfHam]

I also have this problem

[EamonnMR]

Is there a workaround to fix the app on a mac after we unzip it?

[akien-mga]

Use 3.2.3-stable.

[sszigeti]

Just exported a game for Linux on Windows10 with 3.3.stable, and it didn't add the +x permission to the executable.

Also, perhaps unrelated, but the exported Mac version (exported from a Mac with 3.3.stable) doesn't start using the downloaded .dmg, but the same, local .dmg starts without issues.

[sairam4123]

Also, perhaps unrelated, but the exported Mac version (exported from a Mac with 3.3.stable) doesn't start using the downloaded .dmg, but the same, local .dmg starts without issues.

Any problems with that, my friend also has the similar problem do I have to use 3.2.3.stable?

[bruvzg]

Any problems with that, my friend also has the similar problem do I have to use 3.2.3.stable?

Is it signed? Unsigned macOS apps won't run when downloaded from unknown source, until quarantine file attribute is removed (use xattr -dr com.apple.quarantine /path/to/Application.app command after downloading). On Apple Silicon macs unsigned apps won't run at all.

[sairam4123]

No it's not signed, removing the com.apple.quarantine worked! Thanks!

[bruvzg]

Currently, the situation for macOS exports looks like this:

• App is signed with Apple issues signature and notarized - it will display warning dialog on the first start and run without issues.
• App is ad-hoc signed (can be done by selecting Codesign/Enabled in the export options, and setting Codesign/Identity to -) - app won't run, but you can unlock it with right-clicking app in the Finder and selecting "Open" two times in the row.
• App is not signed - It won't run and can be unlocked using the aforementioned command, for older Godot versions without Apple Silicon support right-clicking method might also work.

See https://disable-gatekeeper.github.io/ for some extra options.

[Anutrix]

Some of bug comments here might have been fixed with #62616 in master.

Also, wasn't this ticket initially about +x attribute but now is blocked only by gatekeeper stuff? Should be close and make a new ticket for this?

Also, Gatekeeper situation can only solved completely by Apple notarization.
Possible actions that can be taken/discussed for Gatekeeper:

• Some doc notes mentioning is Gatekeeper related sources/solutions can be added(including bruvg's comment).
• If possible, do it apparently how Blender does it.

[aaronfranke]

@Anutrix No, I can confirm that on macOS 12.4 with a recent Godot master build that it's not setting executable permission on export. When I export the .app and try to run, macOS says "The application “myapp” can’t be opened". If I run chmod -R +x myapp.app then it works fine. This issue is still present, Godot isn't ensuring the app is executable.

This only happens on macOS when exporting a .app or a .dmg, but if I export a .zip and extract it manually then it will have the correct executable permissions and it runs fine.

[akien-mga]

That was a regression in alpha 11, that's fixed already.

[voylin]

@Calinou Should this not have the label of platform:macos instead of Linuxbsd?
On Linux with Godot 4.1.1, export are executable on export for me.

[akien-mga]

Discussed this briefly with @bruvzg, this should be resolved as far as technically feasible by now (probably since at least 4.0, but there might have been further improvements in 4.1 so I'll mark it as fixed in 4.1).

The executable permissions for Linux and macOS binaries are set in all cases where it's possible.

Notably on Windows, which doesn't have a concept of file permissions, the permissions can only be set when doing an export as .zip, which exports a zip with the game binary and PCK together, and sets the permissions in the zip. These permissions should work as long as you don't unzip it and rezip it manually on Windows.

Note that the UX to do a zip export isn't great currently, for example for a Linux build you should do a normal export as usual, and replace the gamename.x86_64 by gamename.zip (you can see that the filter mentions both .x86_64 and .zip as valid extensions).

[Calinou]

Note that the UX to do a zip export isn't great currently, for example for a Linux build you should do a normal export as usual, and replace the gamename.x86_64 by gamename.zip (you can see that the filter mentions both .x86_64 and .zip as valid extensions).

To clarify, this is not the same as using the Export PCK/ZIP option. You need to use the standard Export Project button instead.