Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[NeMo-UX] Make TE and Apex dependencies optional #9550

Merged
merged 48 commits into from
Jul 15, 2024
Merged

[NeMo-UX] Make TE and Apex dependencies optional #9550

merged 48 commits into from
Jul 15, 2024

Commits on Jul 5, 2024

  1. Provide a pure pytorch/jit path to avoid required dependency on TE an… 

    …d Apex

Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    4c0711e View commit details
    Browse the repository at this point in the history

  2. add missing file 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    9877957 View commit details
    Browse the repository at this point in the history

  3. add minimal gpt pretraining example 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    18b8725 View commit details
    Browse the repository at this point in the history

  4. fix pre-training datamodule initialization 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    7d3ee9e View commit details
    Browse the repository at this point in the history

  5. add non-te/non-apex test 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    7a15ec6 View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2024

  1. add comment to pretraining script 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 6, 2024
    Configuration menu
    Copy the full SHA
    72a0f32 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Merge branch 'main' of github.com:NVIDIA/NeMo into ashors/remove-te-a… 

    …pex-deps
    @ashors1
    ashors1 committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    f915988 View commit details
    Browse the repository at this point in the history

  2. use microbatch calculator from mcore 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    12c1cf6 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. Merge branch 'r2.0.0rc1' of github.com:NVIDIA/NeMo into ashors/remove… 

    …-te-apex-deps
    @ashors1
    ashors1 committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    7ef94a1 View commit details
    Browse the repository at this point in the history

  2. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
    @ashors1
    ashors1 committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    2cf377c View commit details
    Browse the repository at this point in the history

  3. fix nemo 2 test name 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    f8ca7c0 View commit details
    Browse the repository at this point in the history

  4. update Mcore commit for CI 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    fdc6ecd View commit details
    Browse the repository at this point in the history

  5. replace apex microbatch calculator with megatron's in more places 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    11279d7 View commit details
    Browse the repository at this point in the history

  6. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
    @ashors1
    ashors1 committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    925dcbc View commit details
    Browse the repository at this point in the history

  7. fix missing import 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    95aac74 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. fix typo 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    ca56e7c View commit details
    Browse the repository at this point in the history

  2. fix missed apex import 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    5e1c95c View commit details
    Browse the repository at this point in the history

  3. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    19cc86d View commit details
    Browse the repository at this point in the history

  4. move imports 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    be9a536 View commit details
    Browse the repository at this point in the history

  5. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    8c03e68 View commit details
    Browse the repository at this point in the history

  6. move imports 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    b3afddf View commit details
    Browse the repository at this point in the history

  7. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    bdc11f5 View commit details
    Browse the repository at this point in the history

  8. add types to command-line args 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    6e45a2d View commit details
    Browse the repository at this point in the history

  9. bug fix 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    97270b8 View commit details
    Browse the repository at this point in the history

  10. fix path 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    5296cc8 View commit details
    Browse the repository at this point in the history

  11. Disable distributed optimizer in nemo 2.0 test 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    7b20d6c View commit details
    Browse the repository at this point in the history

  12. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    83644d1 View commit details
    Browse the repository at this point in the history

  13. fix optimizer config 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    2c1c46b View commit details
    Browse the repository at this point in the history

  14. update checkpointing 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    dabe1d9 View commit details
    Browse the repository at this point in the history

  15. move import 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    c6e1621 View commit details
    Browse the repository at this point in the history

  16. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    317bb41 View commit details
    Browse the repository at this point in the history

  17. fix failing unit test 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    b32b355 View commit details
    Browse the repository at this point in the history

  18. Merge branch 'r2.0.0rc1' into ashors/remove-te-apex-deps

    @ericharper
    ericharper authored Jul 10, 2024
    Configuration menu
    Copy the full SHA
    f90539c View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. fix failing test 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    783063d View commit details
    Browse the repository at this point in the history

  2. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
    @ashors1
    ashors1 committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    8a3e1d4 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'r2.0.0rc1' of github.com:NVIDIA/NeMo into ashors/remove… 

    …-te-apex-deps
    @ashors1
    ashors1 committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    1176d52 View commit details
    Browse the repository at this point in the history

  4. Updating num_weights check of RETRO due to underlying changes from mc… 

    …ore RETRO MLM

Signed-off-by: huvunvidia <86480512+huvunvidia@users.noreply.github.com>
    @huvunvidia @ashors1
    huvunvidia authored and ashors1 committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    d965e74 View commit details
    Browse the repository at this point in the history

  5. Apply isort and black reformatting 

    Signed-off-by: huvunvidia <huvunvidia@users.noreply.github.com>
    @huvunvidia @ashors1
    huvunvidia authored and ashors1 committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    3b33b5b View commit details
    Browse the repository at this point in the history

  6. fix typo 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    c0f3b0d View commit details
    Browse the repository at this point in the history

  7. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
    @ashors1
    ashors1 committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    730a67f View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. Merge branch 'r2.0.0rc1' of github.com:NVIDIA/NeMo into ashors/remove… 

    …-te-apex-deps
    @ashors1
    ashors1 committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    de83795 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'ashors/remove-te-apex-deps' of github.com:NVIDIA/NeMo i… 

    …nto ashors/remove-te-apex-deps
    @ashors1
    ashors1 committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    36ae19e View commit details
    Browse the repository at this point in the history

  3. Merge branch 'r2.0.0rc1' into ashors/remove-te-apex-deps

    @ashors1
    ashors1 authored Jul 12, 2024
    Configuration menu
    Copy the full SHA
    2f42db7 View commit details
    Browse the repository at this point in the history

  4. remove stale warning 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    29058aa View commit details
    Browse the repository at this point in the history

  5. fix lora notebook 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    349a51c View commit details
    Browse the repository at this point in the history

  6. fix small typo 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    5e56a8e View commit details
    Browse the repository at this point in the history

  7. add import guards to gemma2 

    Signed-off-by: ashors1 <ashors@nvidia.com>
    @ashors1
    ashors1 committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    b81d3d9 View commit details
    Browse the repository at this point in the history

  8. Apply isort and black reformatting 

    Signed-off-by: ashors1 <ashors1@users.noreply.github.com>
    @ashors1
    ashors1 committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    d1a1e87 View commit details
    Browse the repository at this point in the history