Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Weights sharding for Keras saving #19286

Draft
wants to merge 7 commits into
base: master
Choose a base branch
from
Draft

Weights sharding for Keras saving #19286

wants to merge 7 commits into from

Commits on Dec 6, 2023

  1. Add initial commit for sharded H5 store

    @nkovela1
    nkovela1 committed Dec 6, 2023
    Configuration menu
    Copy the full SHA
    4058a18 View commit details
    Browse the repository at this point in the history

Commits on Mar 8, 2024

  1. Finish ShardedH5IOStore initial implementation

    @nkovela1
    nkovela1 committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    51ea848 View commit details
    Browse the repository at this point in the history

  2. Add sharding to saving and loading API logic with associated errors

    @nkovela1
    nkovela1 committed Mar 8, 2024
    Configuration menu
    Copy the full SHA
    a7d29aa View commit details
    Browse the repository at this point in the history

Commits on Mar 9, 2024

  1. Fix sharding API and add size check

    @nkovela1
    nkovela1 committed Mar 9, 2024
    Configuration menu
    Copy the full SHA
    de397ed View commit details
    Browse the repository at this point in the history

Commits on Mar 11, 2024

  1. Add large model test and debug sharding algorithm

    @nkovela1
    nkovela1 committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    933c387 View commit details
    Browse the repository at this point in the history

  2. Merge update with master saving changes

    @nkovela1
    nkovela1 committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    ad7ca7d View commit details
    Browse the repository at this point in the history

  3. Fix formatting

    @nkovela1
    nkovela1 committed Mar 11, 2024
    Configuration menu
    Copy the full SHA
    cfbb761 View commit details
    Browse the repository at this point in the history