Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Keras Core (Keras 3.0) support #26224

Closed
wants to merge 3 commits into from
Closed

Conversation

Rocketknight1
Copy link
Member

@Rocketknight1 Rocketknight1 commented Sep 18, 2023

This PR is a very-very-very work in progress effort to add Keras Core support, and to prepare for the transition to Keras 3.0. Because Keras Core is still a beta/preview, this PR is likely to be fairly slow and cautious, we don't want to lock ourselves into an API that may change under our feet!

The goal of this PR is to create a "minimum viable port(duct)" of our tf.keras code to keras-core to assess how difficult it will be to support Keras Core. Therefore, this port has the following properties:

  • Only loading from safetensors will be supported for now, so I don't have to support every possible combination of (weights_format, model_framework)
  • All mixins like GenerationMixin will be inherited directly from the framework that corresponds to the active keras-core framework. This means that keras-core classes will change their inheritance depending on which framework is live, which is risky but greatly reduces the amount of code I need to write.
  • We will mainly be supporting TF and JAX as Keras Core frameworks. All of our models have PyTorch code already, which will probably be more stable and better-tested than using Keras Core + PyTorch, so I doubt we'd see much usage there. In addition, TF and JAX fit Keras Core's assumptions much more naturally than PyTorch does.

The plan for this PR is:

  • Create modeling_keras_outputs.py (ported from modeling_tf_outputs.py)
  • Create modeling_keras_utils.py (ported from modeling_tf_utils.py)
  • Port a single model to keras-core (probably BERT or DistilBERT)
  • Add model tests to ensure that outputs match
  • Improve support for our data classes, like supporting BatchEncoding in fit() or auto-wrapping HF datasets in a Keras PyDataset

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

@Rocketknight1
Copy link
Member Author

Don't stale yet!

@huggingface huggingface deleted a comment from github-actions bot Oct 19, 2023
Copy link

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@ariG23498
Copy link
Contributor

Hey @Rocketknight1 I would like to work on this. Is HF still looking forward to have Keras 3.0 integrated?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants