Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hard error when ignoring tensors. (#27484) #29906

Merged
merged 13 commits into from
Apr 2, 2024
Merged

Commits on Mar 27, 2024

  1. Hard error when ignoring tensors. (#27484)

    * [WIP] Hard error when ignoring tensors.
    
    * Better selection/error when saving a checkpoint.
    
    - Find all names we should normally drop (those are in the transformers
      config)
    - Find all disjoint tensors (for those we can safely trigger a copy to
      get rid of the sharing before saving)
    - Clone those disjoint tensors getting rid of the issue
    - Find all identical names (those should be declared in the config
      but we try to find them all anyway.)
    - For all identical names:
      - If they are in the config, just ignore them everything is fine
      - If they are not, warn about them.
    - For all remainder tensors which are shared yet neither identical NOR
      disjoint. raise a hard error.
    
    * Adding a failing test on `main` that passes here.
    
    * We don't need to keep the subfolder logic in this test.
    
    * Apply suggestions from code review
    
    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    
    ---------
    
    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    Narsil and ArthurZucker committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    e7e3290 View commit details
    Browse the repository at this point in the history
  2. Add small tests.

    Narsil committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    528ad72 View commit details
    Browse the repository at this point in the history
  3. Dead variable.

    Narsil committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    5c4aaf5 View commit details
    Browse the repository at this point in the history
  4. Fixup.

    Narsil committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    270e6ea View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    7cd1593 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    d561bef View commit details
    Browse the repository at this point in the history
  7. Code quality.

    Narsil committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    e5bec8f View commit details
    Browse the repository at this point in the history
  8. Dynamic member.

    Narsil committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    71f1f67 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. trigger

    ydshieh committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    d915cc5 View commit details
    Browse the repository at this point in the history

Commits on Mar 29, 2024

  1. Configuration menu
    Copy the full SHA
    e7a3186 View commit details
    Browse the repository at this point in the history
  2. Fix scoping.

    Narsil committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    455b478 View commit details
    Browse the repository at this point in the history
  3. Update .github/workflows/self-scheduled.yml

    Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
    Narsil and ArthurZucker authored Mar 29, 2024
    Configuration menu
    Copy the full SHA
    a053ec7 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    7974f97 View commit details
    Browse the repository at this point in the history