Adapter appears to not train

[gpucce]

Hi, I am trying to replicate the results, however it looks like if I save the adapter after training using the command python pipeline_adapter.py fingerprint --base_model Mistral-7B-v0.1 which is done already and goes to instruction_emb.pt and I save it before training happens, the two saved file appear to be identical, and both A and B only have zeros.

And indeed if I try to load the saved adapter on a new model it generates more or less gibberish values.