Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial-exec TLS variables do not work in the first thread #810

Open
nyh opened this issue Nov 3, 2016 · 1 comment
Open

Initial-exec TLS variables do not work in the first thread #810

nyh opened this issue Nov 3, 2016 · 1 comment

Comments

@nyh
Copy link
Contributor

nyh commented Nov 3, 2016

@benoit-canet discovered a bug in the way we run the init functions of a shared objects which has initial-exec (aka "static") TLS variables (and those are used by the init functions). This effects, for example, golang (see #522) programs, which uses both. But that is part of a more general problem with how we set up static TLS:

The program::get_library() function does not, and should not, create a new thread (only the osv::application wrapper does). One can load a library and call a function from it in the current thread, without creating a new thread.

The problem is that get_library() calls init_static_tls() which only changes the way that new threads will be created (when they will call setup_tcb()) - and does not modify the current thread's TLS area.

So in a program which uses static TLS (the initial-exec model), and runs code from the library without starting a new thread, it will not have access to its TLS variables.

In particular (and this is what led @benoit-canet to discover this issue), currently even using the osv::application API (which does start new threads), something is broken: The shared-object's init functions are run on the original thread, so they cannot access the TLS variables.
As a workaround, @benoit-canet proposed a patch where osv::application first sets up static TLS (which will effect only new threads), then creates the new thread, and then, in it, runs the initialization functions.

I think it should be possible to fix the underlying problem, i.e., to fix init_static_tls() to resize and rebuild the current thread's static TLS area - not just set up the one for new threads. If we would do that, we won't need to modify the application creation code to split the loading and initialization parts. Moreover, things would work even if we chose not to create a new thread at all. We also need to consider what happens when the library is unloaded (we may be unable to reduce the TLS area's size and need to leak memory?).

@nyh nyh changed the title initial-exec TLS woes Initial-exec TLS variables do not work in the first thread Feb 5, 2017
@nyh
Copy link
Contributor Author

nyh commented Feb 5, 2017

If we want to support initial-exec in a shared object loaded by program::get_library() - and not just a new app returned by osv::application::run(), we also need to note this:

thread::setup_tcb() sets up the initial-exec area for new threads. It needs a list of shared objects to include in the initial-exec TLS area for the new thread, and finds this list by looking at the current thread's app_runtime object and look at its initial_tls(). That was set up by object::init_static_tls() and only included objects loaded by dependencies, not get_library(). So get_library() needs to update _initial_tls of the object of the current thread's app_runtime (similar to init_static_tls()) and then use that to rebuild the tcb (similar to setup_tcb but without destroying already existing data). We probably also need to update _tls for accessing the same data through non-initial-exec tls methods. All of this is pretty complex to do properly...

@benoit-canet 's workaround of supporting initial-exec only for osv::application::run() by running the library's init functions only in the child thread is starting to look appealing... If we do this, we should at least find a way of warning when loading a shared object with initial-exec variables in an unsupported way....

nyh pushed a commit that referenced this issue Apr 15, 2018
This patch addresses two issues related to Golang support on OSv.

Firstly it provides a workaround around the issue described by #810.
In essence some applications like Golang runtime during initialization
of it's ELF object invoke init functions that access variables from TLS
(Thread Local Storage) memory area by static offset determined during
compilation. This type of TLS access is called "initial exec"
(or static) and is only supported on OSv if shared library runs
on new thread. To address this limitation this patch introduces
changes that allow delaying ELF initialization until new thread is run.

Secondly this patch provides changes to ELF initialization
code that allow passing argv and other data to shared
object init functions as explained by #795. This is for example
required by Golang runtime.

Fixes #795

The content of this patch was authored by Benoit Canet.

Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Message-Id: <1523160143-6976-1-git-send-email-jwkozaczuk@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant