Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Simplify the Transformer API #1024

Closed
sararb opened this issue Mar 21, 2023 · 0 comments · Fixed by #1022
Closed

[FEA] Simplify the Transformer API #1024

sararb opened this issue Mar 21, 2023 · 0 comments · Fixed by #1022
Labels
area/session-based enhancement New feature or request
Milestone

Comments

@sararb
Copy link
Contributor

sararb commented Mar 21, 2023

🚀 Feature request

The current API provides support for masking and inference through specialized Merlin Blocks, such as ReplaceMaskedEmbeddings and SequenceMaskLastInference. These blocks need to be explicitly set by the user in the pre/post arguments of the TransformerBlock. However, this adds complexity to the high-level MM API, as the user needs to be familiar with these custom blocks in order to correctly define and train a transformer-based model using CLM or MLM approaches.

It would be helpful to have a more streamlined approach for incorporating masking and inference support into transformer models, without requiring users to have extensive knowledge of specialized Merlin Blocks.

@sararb sararb added enhancement New feature or request area/session-based labels Mar 21, 2023
@sararb sararb added this to the Merlin 23.03 milestone Mar 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/session-based enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant