Fix retriever only training

[SachiraKuruppu]

• Use bits and bytes to reduce model size.
• move PEFT config inside the model to be consistent with the e2e RAG.
• Remove hardcoded values. Add command line arguments to make the script generic. Be able to train with different datasets.
• Reduce tokenizer max length to 128. Our current data is less than 100 words. (Maybe this should also be configurable via command line).

[shamanez]

@rsachira could you also add the issue with the padding = True for the future references . Would super helpful if you can add a two sentence summary of our convo. :)

[SachiraKuruppu]

@shamanez This is what I remember from our conversation. Correct me if I'm wrong.

In the tokenizer, setting the padding to max_length causes it to set all the examples to the same length. The downside is we may not need so many tokens to encode the passages.

result_ = tokenizer(queries, padding="max_length", max_length=128, truncation=True)

We can let the tokenizer dynamically select the token size, and pad to the longest length in the batch. This can be achieved by setting the padding to True.

result_ = tokenizer(queries, padding=True, max_length=128, truncation=True)

However, this is not a good approach for us when it comes to in-batch negative contrastive learning, because it may order the input IDs according to the length.

[shamanez]

Remove this line. You don't need this

[SachiraKuruppu]

Done