Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thread safety? #380

Closed
AmirAktify opened this issue Jul 1, 2022 · 3 comments
Closed

Thread safety? #380

AmirAktify opened this issue Jul 1, 2022 · 3 comments
Labels
question Further information is requested Stale

Comments

@AmirAktify
Copy link

I just wanted to ask about inference on adapters and thread safety.
If there are multiple adapters on the model, and different requests are trying to set different adapters active for inference, I expect this would not be threadsafe?

E.g
Thread 1:
model.set_active_adapters('adapter_a")
model.forward(x)...

Thread 2:
model.set_active_adapters('adapter_b')
model.forward(x)

I.e, how those two threads interleave could cause unpredictable outputs?

@AmirAktify AmirAktify added the question Further information is requested label Jul 1, 2022
@calpt
Copy link
Member

calpt commented Jul 11, 2022

Hey, I believe this is related to the discussion in #165 and the solution in #257, which introduces the option to use the AdapterSetup context manager instead of set_active_adapters() (which was released with v3.0.0). Please correct me if this is not what you were asking about.

@adapter-hub-bert
Copy link
Member

This issue has been automatically marked as stale because it has been without activity for 90 days. This issue will be closed in 14 days unless you comment or remove the stale label.

@adapter-hub-bert
Copy link
Member

This issue was closed because it was stale for 14 days without any activity.

@adapter-hub-bert adapter-hub-bert closed this as not planned Won't fix, can't repro, duplicate, stale Oct 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested Stale
Projects
None yet
Development

No branches or pull requests

3 participants