add_embeddings doesn't affect lm_head of t5 model

[nzw0301]

Environment info

• adapter-transformers version: 3.2.1
• Platform: MacOS
• Python version: 3.10.11
• PyTorch version (GPU?): 2.0.0
• Tensorflow version (GPU?): NA
• Using GPU in script?: No
• Using distributed or parallel set-up in script?: No

Information

Model I am using (Bert, XLNet ...): "t5-small"

Language I am using the model on (English, Chinese ...): English

Adapter setup I am using (if any):

The problem arises when using:

• the official example scripts: (give details below)
• my own modified scripts: (give details below)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)
• my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

1. See the code below

from transformers import AutoTokenizer
from transformers import AutoTokenizer
from transformers import AutoModelForSeq2SeqLM


model_name = "t5-small"

tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.add_tokens(["NEW_ADDED_TOKEN"])
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
print(f"original: encoder embedding: {model.encoder.embed_tokens.num_embeddings} decoder embedding: {model.decoder.embed_tokens.num_embeddings} lm_head: {model.lm_head.out_features}")

model.resize_token_embeddings(len(tokenizer))
print(f"resize_ver: encoder embedding: {model.encoder.embed_tokens.num_embeddings} decoder embedding: {model.decoder.embed_tokens.num_embeddings} lm_head: {model.lm_head.out_features}")

reference_tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

model.add_embeddings(
    name="adapter_embedding",
    tokenizer=tokenizer,
    reference_embedding="default",
    reference_tokenizer=reference_tokenizer
)

print(f"add_embeddings: encoder embedding: {model.encoder.embed_tokens.num_embeddings} decoder embedding: {model.decoder.embed_tokens.num_embeddings} lm_head: {model.lm_head.out_features}")

The outputs are:

original: encoder embedding: 32128 decoder embedding: 32128 lm_head: 32128
resize_ver: encoder embedding: 32101 decoder embedding: 32101 lm_head: 32101
add_embeddings: encoder embedding: 32101 decoder embedding: 32101 lm_head: 32128

Expected behavior

The output of the last line should be add_embeddings: encoder embedding: 32101 decoder embedding: 32101 lm_head: 32101. In my understanding, a T5 model shares encoder's embedding, decoder embeddings and lm_head. This can be checked by seeing all(model.encoder.embed_tokens.weight[0] == model.decoder.embed_tokens.weight[0]), all(model.encoder.embed_tokens.weight[0] == model.lm_head.weight[0]). However, as shown above added_embeddings method does not affect model.lm_head. This would be more consistent with resize_token_embeddings's result.