How to call Python script from C++ code within Gramine

[hello31337]

Hello,

I'm trying to call a Python script with the system() function from C++ code inside Gramine, after completing Remote Attestation in C++ code like ra-tls-mbedtls.
I tried the following configuration, but 32512 (0x7F00, equivalent to error 127) was returned as the return value of the system function.
I assume this is an error due to not being able to locate python, but is there a good way to call python within Gramine?
I'm using ra-tls-mbedtls as a base and the RA execution and TLS communication itself is successful.

• caller

pyret = std::system("./python helloworld.py");
std::cout << pyret << std::endl;

• Manifest part of Makefile

server.manifest: server.manifest.template
	gramine-manifest \
		-Dlog_level=$(GRAMINE_LOG_LEVEL) \
		-Darch_libdir=$(ARCH_LIBDIR) \
		-Dra_type=$(RA_TYPE) \
		-Dra_client_spid=$(RA_CLIENT_SPID) \
		-Dra_client_linkable=$(RA_CLIENT_LINKABLE) \
		-Dpython_loc=$(realpath $(shell sh -c "command -v python3")) \
		$< > $@

• Manifest (server.manifest.template)

# Copyright (C) 2023 Gramine contributors
# SPDX-License-Identifier: BSD-3-Clause

# RA-TLS manifest file example

loader.entrypoint = "file:{{ gramine.libos }}"
libos.entrypoint = "/server"

loader.log_level = "{{ log_level }}"

loader.env.LD_LIBRARY_PATH = "/lib:{{ arch_libdir }}:/usr{{ arch_libdir }}"

loader.insecure__use_cmdline_argv = true

sys.enable_sigterm_injection = true

fs.mounts = [
  { path = "/lib", uri = "file:{{ gramine.runtimedir() }}" },
  { path = "{{ arch_libdir }}", uri = "file:{{ arch_libdir }}" },
  { path = "/usr{{ arch_libdir }}", uri = "file:/usr{{ arch_libdir }}" },
{% for path in python.get_sys_path(python_loc) %}
  { path = "{{ path }}", uri = "file:{{ path }}" },
{% endfor %}
  { path = "{{ python_loc }}", uri = "file:{{ python_loc }}" },
  { path = "/etc", uri = "file:/etc" },
  { path = "/server", uri = "file:server" },

  { type = "tmpfs", path = "/tmp" },
]

sgx.debug = false
sgx.edmm_enable = {{ 'true' if env.get('EDMM', '0') == '1' else 'false' }}

sgx.remote_attestation = "{{ ra_type }}"
sgx.ra_client_spid = "{{ ra_client_spid }}"
sgx.ra_client_linkable = {{ 'true' if ra_client_linkable == '1' else 'false' }}

sgx.trusted_files = [
  "file:{{ gramine.libos }}",
  "file:server",
  "file:{{ gramine.runtimedir() }}/",
  "file:{{ arch_libdir }}/",
  "file:/usr{{ arch_libdir }}/",
  "file:ssl/ca.crt",
  "file:ssl/server.crt",
  "file:ssl/server.key",
  "file:{{ python_loc }}",
{% for path in python.get_sys_path(python_loc) %}
  "file:{{ path }}{{ '/' if path.is_dir() else '' }}",
{% endfor %}
  "file:helloworld.py",
]

sgx.allowed_files = [
  "file:/etc/nsswitch.conf",
  "file:/etc/ethers",
  "file:/etc/hosts",
  "file:/etc/group",
  "file:/etc/passwd",
  "file:/etc/gai.conf",
]

[kailun-qin]

@hello31337 Thanks for your question!

In your case, the shell (that system() executes underneath), the python executable and helloworld.py are required but currently missing in the manifest. I think you'll need to specify them in the trusted files as well as in the FS mounts.

[hello31337]

Thank you! Finally, I set up the following configuration and succeeded in running Python from C++.

Part of the manifest:

fs.mounts = [
  { path = "/lib", uri = "file:{{ gramine.runtimedir() }}" },
  { path = "{{ arch_libdir }}", uri = "file:{{ arch_libdir }}" },
  { path = "/usr{{ arch_libdir }}", uri = "file:/usr{{ arch_libdir }}" },
{% for path in python.get_sys_path(python_loc) %}
  { path = "{{ path }}", uri = "file:{{ path }}" },
{% endfor %}
  { path = "{{ python_loc }}", uri = "file:{{ python_loc }}" },
  { path = "/etc", uri = "file:/etc" },
  { path = "/server", uri = "file:server" },
  { path = "/python3", uri = "file:/usr/bin/python3" },
  { path = "/bin/sh", uri = "file:/bin/sh" },

  { type = "tmpfs", path = "/tmp" },
]

sgx.trusted_files = [
  "file:{{ gramine.libos }}",
  "file:server",
  "file:{{ gramine.runtimedir() }}/",
  "file:{{ arch_libdir }}/",
  "file:/usr{{ arch_libdir }}/",
  "file:ssl/ca.crt",
  "file:ssl/server.crt",
  "file:ssl/server.key",
  "file:{{ python_loc }}",
{% for path in python.get_sys_path(python_loc) %}
  "file:{{ path }}{{ '/' if path.is_dir() else '' }}",
{% endfor %}
  "file:/usr/bin/python3",
  "file:/bin/sh",
  "file:helloworld.py",
]

Caller:

pyret = std::system("/python3 helloworld.py 100 200");

Please let me ask one more question.

It seems to me that files and folders registered in fs.mounts and trusted_files (including shells, Python and helloworld.py) need to be assumed to be trusted in order to run within Gramine-SGX.
Therefore, is my understanding correct that the process (./server) is to build in a trusted environment, such as a remote client's environment, then send the resulting binary to the untrusted SGX server that actually runs Gramine, and then check it with RA?

Sorry for my limited understanding of the security model in Gramine.

[mkow]

Therefore, is my understanding correct that the process (./server) is to build in a trusted environment, such as a remote client's environment, then send the resulting binary to the untrusted SGX server that actually runs Gramine, and then check it with RA?

Yup. Usually you just bundle and ship everything together (app and all dependent files, manifest, Gramine), run on the server and attest remotely that the hash matches the expected one, before sending any secret data for the workload.

[hello31337]

Thank you! Your advice was very helpful!
If I have any more questions I will ask again.