Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sequence parallel with communication overlap #5691

Merged
merged 20 commits into from
Aug 1, 2024
Merged

sequence parallel with communication overlap #5691

merged 20 commits into from
Aug 1, 2024

Commits on May 21, 2024

  1. fix ds-sp grad scale for zero0

    @inkcherry
    inkcherry committed May 21, 2024
    Configuration menu
    Copy the full SHA
    cb15ffa View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. enable o compute async

    @inkcherry
    inkcherry committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    a037a53 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. enable qk bwd async all2all

    @inkcherry
    inkcherry committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    42d1284 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. fwd optimi

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    6919af4 View commit details
    Browse the repository at this point in the history

  2. fix1 remove linear arg, remove note

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    39596ac View commit details
    Browse the repository at this point in the history

  3. async qkv fwd, optimi cpu ,make fwd call fast

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    eb760c0 View commit details
    Browse the repository at this point in the history

  4. update

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    c7d3374 View commit details
    Browse the repository at this point in the history

  5. refine code

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    70a6d0c View commit details
    Browse the repository at this point in the history

  6. refine code

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    65afd89 View commit details
    Browse the repository at this point in the history

  7. Revert "fix ds-sp grad scale for zero0" 

    This reverts commit cb15ffa.
    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    4b3518e View commit details
    Browse the repository at this point in the history

  8. Merge remote-tracking branch 'upstream/master' into sp_overlap_comm

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    634d6d9 View commit details
    Browse the repository at this point in the history

  9. fix format

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    54b5ce3 View commit details
    Browse the repository at this point in the history

  10. fix format

    @inkcherry
    inkcherry committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    c9f0c0a View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. refine code

    @inkcherry
    inkcherry committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    0862aa3 View commit details
    Browse the repository at this point in the history

  2. add register for v, ensuring they launch on a single thread.

    @inkcherry
    inkcherry committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    1c596dd View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Merge branch 'master' into sp_overlap_comm

    @tjruwase
    tjruwase authored Jul 8, 2024
    Configuration menu
    Copy the full SHA
    96e7696 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. remove v

    @inkcherry
    inkcherry committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    2fbbd5e View commit details
    Browse the repository at this point in the history

  2. remove v 

    remove v
    @inkcherry
    inkcherry authored Jul 10, 2024
    Configuration menu
    Copy the full SHA
    765a664 View commit details
    Browse the repository at this point in the history

  3. fix notes and format

    @inkcherry
    inkcherry committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    171eb67 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. Merge branch 'master' into sp_overlap_comm

    @HeyangQin
    HeyangQin authored Jul 12, 2024
    Configuration menu
    Copy the full SHA
    020ab5f View commit details
    Browse the repository at this point in the history