You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I just wanted to ask about inference on adapters and thread safety.
If there are multiple adapters on the model, and different requests are trying to set different adapters active for inference, I expect this would not be threadsafe?
Hey, I believe this is related to the discussion in #165 and the solution in #257, which introduces the option to use the AdapterSetup context manager instead of set_active_adapters() (which was released with v3.0.0). Please correct me if this is not what you were asking about.
This issue has been automatically marked as stale because it has been without activity for 90 days. This issue will be closed in 14 days unless you comment or remove the stale label.
I just wanted to ask about inference on adapters and thread safety.
If there are multiple adapters on the model, and different requests are trying to set different adapters active for inference, I expect this would not be threadsafe?
E.g
Thread 1:
model.set_active_adapters('adapter_a")
model.forward(x)...
Thread 2:
model.set_active_adapters('adapter_b')
model.forward(x)
I.e, how those two threads interleave could cause unpredictable outputs?
The text was updated successfully, but these errors were encountered: