Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[NeMo-UX] Add PEFT #9490

Merged
merged 48 commits into from
Jul 5, 2024
Merged

[NeMo-UX] Add PEFT #9490

merged 48 commits into from
Jul 5, 2024

Commits on Jul 4, 2024

  1. initial commit for PEFT in nemo2

    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    a140dcf View commit details
    Browse the repository at this point in the history

  2. Apply isort and black reformatting 

    Signed-off-by: cuichenx <cuichenx@users.noreply.github.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    a62fd06 View commit details
    Browse the repository at this point in the history

  3. address comments 

    Signed-off-by: Chen Cui <chcui@nvidia.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    76b23be View commit details
    Browse the repository at this point in the history

  4. make import easier 

    Signed-off-by: Chen Cui <chcui@nvidia.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    c87896b View commit details
    Browse the repository at this point in the history

  5. Apply isort and black reformatting 

    Signed-off-by: cuichenx <cuichenx@users.noreply.github.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    e356a1f View commit details
    Browse the repository at this point in the history

  6. address comments 

    Signed-off-by: Chen Cui <chcui@nvidia.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    0d649de View commit details
    Browse the repository at this point in the history

  7. Update nemo/collections/llm/peft/lora.py 

    Signed-off-by: Marc Romeyn <marcromeyn@gmail.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    6b98061 View commit details
    Browse the repository at this point in the history

  8. Some small fixes + adding more doc-strings

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    76f3a48 View commit details
    Browse the repository at this point in the history

  9. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    1b92d15 View commit details
    Browse the repository at this point in the history

  10. Adding ModelTransform callback

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    21dc1c5 View commit details
    Browse the repository at this point in the history

  11. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    6666535 View commit details
    Browse the repository at this point in the history

  12. Fixing type-hint for model_transform

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    38cb190 View commit details
    Browse the repository at this point in the history

  13. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3d8351f View commit details
    Browse the repository at this point in the history

  14. fix import 

    Signed-off-by: Chen Cui <chcui@nvidia.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    741c998 View commit details
    Browse the repository at this point in the history

  15. model transform for gemma llama 

    Signed-off-by: Chen Cui <chcui@nvidia.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    ac4eec1 View commit details
    Browse the repository at this point in the history

  16. Apply isort and black reformatting 

    Signed-off-by: cuichenx <cuichenx@users.noreply.github.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    eadf9fe View commit details
    Browse the repository at this point in the history

  17. fix model transform 

    Signed-off-by: Chen Cui <chcui@nvidia.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3228035 View commit details
    Browse the repository at this point in the history

  18. Apply isort and black reformatting 

    Signed-off-by: cuichenx <cuichenx@users.noreply.github.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    21a2b10 View commit details
    Browse the repository at this point in the history

  19. change lora target default to all linear modules 

    Signed-off-by: Chen Cui <chcui@nvidia.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    85bfad6 View commit details
    Browse the repository at this point in the history

  20. Apply isort and black reformatting 

    Signed-off-by: cuichenx <cuichenx@users.noreply.github.com>
    @cuichenx @marcromeyn
    cuichenx authored and marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    b64789b View commit details
    Browse the repository at this point in the history

  21. Small fix in mixtral

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    077469e View commit details
    Browse the repository at this point in the history

  22. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    7cd790a View commit details
    Browse the repository at this point in the history

  23. Integrating PEFT to the public-API + some fixes

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    e083e3c View commit details
    Browse the repository at this point in the history

  24. Big refactor to allow to load adapter-states

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    3df37df View commit details
    Browse the repository at this point in the history

  25. Some fixes to support adapter_path

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    ef76686 View commit details
    Browse the repository at this point in the history

  26. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    908feed View commit details
    Browse the repository at this point in the history

  27. Disabling ckpt reloading when adapter_path is passed

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    6c7be3d View commit details
    Browse the repository at this point in the history

  28. Fix CLI

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    f14a3c5 View commit details
    Browse the repository at this point in the history

  29. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    cdabd75 View commit details
    Browse the repository at this point in the history

  30. Remove commented-out code

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    08fb761 View commit details
    Browse the repository at this point in the history

  31. Remove commented-out code

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    7d96473 View commit details
    Browse the repository at this point in the history

  32. Remove un-used import

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    454c23f View commit details
    Browse the repository at this point in the history

  33. Fix callback imports

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    0e43371 View commit details
    Browse the repository at this point in the history

  34. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    aa711bd View commit details
    Browse the repository at this point in the history

  35. Fixing llm.pretrain

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    baca872 View commit details
    Browse the repository at this point in the history

  36. Some small fixes

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    2d12a84 View commit details
    Browse the repository at this point in the history

  37. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    553b3f3 View commit details
    Browse the repository at this point in the history

  38. Fix missing import + type-hint in finetune

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    45690f3 View commit details
    Browse the repository at this point in the history

  39. Adding PreemptionCallback + some more tests

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    835cdd3 View commit details
    Browse the repository at this point in the history

  40. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    e4d809e View commit details
    Browse the repository at this point in the history

  41. Clean up imports & clean up llm.api

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    30b967b View commit details
    Browse the repository at this point in the history

  42. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    2617a15 View commit details
    Browse the repository at this point in the history

  43. Trying to fix failing tests

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    2e9857e View commit details
    Browse the repository at this point in the history

  44. Merge branch 'main' into nemo-ux/peft

    @marcromeyn
    marcromeyn authored Jul 4, 2024
    Configuration menu
    Copy the full SHA
    c80e7b7 View commit details
    Browse the repository at this point in the history

  45. Remove __init__.py 2

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    d541a47 View commit details
    Browse the repository at this point in the history

  46. Apply isort and black reformatting 

    Signed-off-by: marcromeyn <marcromeyn@users.noreply.github.com>
    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    7a153d3 View commit details
    Browse the repository at this point in the history

  47. Fix failing test

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    37b5b73 View commit details
    Browse the repository at this point in the history

  48. Trying to fix last failing test

    @marcromeyn
    marcromeyn committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    eb72dad View commit details
    Browse the repository at this point in the history