Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AdapterSetup context manager #257

Merged
merged 15 commits into from
Jan 19, 2022
Merged

Conversation

calpt
Copy link
Member

@calpt calpt commented Dec 2, 2021

Background

All adapter-supporting model classes currently provide an adapter_names parameter in their forward() call that allows dynamic & state-less configuration of the activated adapters. We tried to remove this parameter with the release of v2.0.0, however it turned out to be essential for a multi-threaded inference use-case (see #165 for context). The removal was therefore reverted shortly after (#176; v.2.0.1).

Problem

The adapter_names parameter has to be passed down all module classes to the actual adapter layers. This means that the method headers of all top-level and intermediate module classes need to be modified which makes maintanance difficult & prevents further disentangling AH & HF.

Suggested solution

As a replacement to the adapter_names parameter, this PR introduces a new AdapterSetup class. This class is intended to be used as a context manager, i.e. a typical use case would look like this:

# will use no adapters
outputs = model(**inputs)

with AdapterSetup(Stack("a", "b")):
    # will use the adapter stack "a" and "b"
    outputs = model(**inputs)

Note that in the above example no adapters are activated via active_adapters. Within the with block, the adapter implementation will read the currently active setup from the context manager.

This solution allows dynamic adapter activation, e.g. also with nesting:

with AdapterSetup(Stack("a", "b")):
    # will use the adapter stack "a" and "b"
    outputs = model(**inputs)
    with AdapterSetup(Fuse("c", "d"), head_setup="e"):
        # will use fusion between "c" and "d" & head "e"
        outputs = model(**inputs)

Most importantly, the context manager is thread-local, i.e. we can use different setups in different threads with the same model instance. There's a test for this:
https://github.com/calpt/adapter-transformers/blob/93065b74ae933b34ec5c3f4047223792a795239f/tests/test_adapter_setup_context.py#L75-L112

Breaking changes

  • The adapter_names parameter is removed for all model classes

src/transformers/adapters/heads/base.py Show resolved Hide resolved
src/transformers/models/bart/modeling_bart.py Outdated Show resolved Hide resolved
adapter_docs/adapter_composition.md Outdated Show resolved Hide resolved
@calpt calpt changed the base branch from master to develop December 10, 2021 14:22
@calpt calpt merged commit 159f6e9 into adapter-hub:develop Jan 19, 2022
@calpt calpt deleted the dev/adapter_context branch January 19, 2022 09:48
calpt added a commit that referenced this pull request Feb 2, 2022
calpt added a commit that referenced this pull request Feb 9, 2022
@calpt calpt mentioned this pull request Jul 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants