Move generation functionality to base classes #1861
Labels
team-created
Issues created by Keras Hub team as part of development roadmap.
type:feature
New feature or request
We want it to be easy to add more generative models to the repository, and we want it to be easy to add new features related to generation.
Currently quite a bit of common functionality for generation live in every
XXCausalLM
subclass, which makes it hard to add new models and build new generation features. I think we can pull a lot of this functionality down into the base class. I also as part of this work we should move the compiled while loop out of the sampler class and into theCausalLM
base class, which will make it much easier to write custom generative loops (e.g. would help integrating with JetStream).The text was updated successfully, but these errors were encountered: