You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current API provides support for masking and inference through specialized Merlin Blocks, such as ReplaceMaskedEmbeddings and SequenceMaskLastInference. These blocks need to be explicitly set by the user in the pre/post arguments of the TransformerBlock. However, this adds complexity to the high-level MM API, as the user needs to be familiar with these custom blocks in order to correctly define and train a transformer-based model using CLM or MLM approaches.
It would be helpful to have a more streamlined approach for incorporating masking and inference support into transformer models, without requiring users to have extensive knowledge of specialized Merlin Blocks.
The text was updated successfully, but these errors were encountered:
🚀 Feature request
The current API provides support for masking and inference through specialized Merlin Blocks, such as
ReplaceMaskedEmbeddings
andSequenceMaskLastInference
. These blocks need to be explicitly set by the user in the pre/post arguments of theTransformerBlock
. However, this adds complexity to the high-level MM API, as the user needs to be familiar with these custom blocks in order to correctly define and train a transformer-based model using CLM or MLM approaches.It would be helpful to have a more streamlined approach for incorporating masking and inference support into transformer models, without requiring users to have extensive knowledge of specialized Merlin Blocks.
The text was updated successfully, but these errors were encountered: