Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dev neil branch creation #23

Open
wants to merge 171 commits into
base: main
Choose a base branch
from
Open

Dev neil branch creation #23

wants to merge 171 commits into from

Commits on Oct 20, 2023

  1. Use HF flash attention and test new hparams

    @ikergarcia1996
    ikergarcia1996 committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    8051dc3 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #1 from hitz-zentroa/main 

    Rebase to main
    @ikergarcia1996
    ikergarcia1996 authored Oct 20, 2023
    Configuration menu
    Copy the full SHA
    dc8d8c1 View commit details
    Browse the repository at this point in the history

  3. Change name

    @ikergarcia1996
    ikergarcia1996 committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    6a857c3 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/dev' into dev

    @ikergarcia1996
    ikergarcia1996 committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    4ce6f63 View commit details
    Browse the repository at this point in the history

  5. Set correct environment

    @ikergarcia1996
    ikergarcia1996 committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    1590daa View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2023

  1. Change hparams

    @ikergarcia1996
    ikergarcia1996 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    cf99869 View commit details
    Browse the repository at this point in the history

  2. 2 GPUs

    @ikergarcia1996
    ikergarcia1996 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    5a10fbe View commit details
    Browse the repository at this point in the history

  3. change project

    @ikergarcia1996
    ikergarcia1996 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    38d51c1 View commit details
    Browse the repository at this point in the history

  4. Fix bug

    @ikergarcia1996
    ikergarcia1996 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    d2df066 View commit details
    Browse the repository at this point in the history

  5. 128

    @ikergarcia1996
    ikergarcia1996 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    2a462ee View commit details
    Browse the repository at this point in the history

  6. Fix name

    @ikergarcia1996
    ikergarcia1996 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    42e95a5 View commit details
    Browse the repository at this point in the history

  7. fix memory fragmentation

    @ikergarcia1996
    ikergarcia1996 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    c9f08f4 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2023

  1. Reduce batch size

    @ikergarcia1996
    ikergarcia1996 committed Oct 22, 2023
    Configuration menu
    Copy the full SHA
    ac41590 View commit details
    Browse the repository at this point in the history

  2. Use 4 GPUs

    @ikergarcia1996
    ikergarcia1996 committed Oct 22, 2023
    Configuration menu
    Copy the full SHA
    d736865 View commit details
    Browse the repository at this point in the history

  3. Increase num threads

    @ikergarcia1996
    ikergarcia1996 committed Oct 22, 2023
    Configuration menu
    Copy the full SHA
    9ab89a8 View commit details
    Browse the repository at this point in the history

  4. Evaluate all checkpoints

    @ikergarcia1996
    ikergarcia1996 committed Oct 22, 2023
    Configuration menu
    Copy the full SHA
    4574408 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. Attempt to fix dtype bug

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    3495f41 View commit details
    Browse the repository at this point in the history

  2. Fix eval scripts

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    4dabec4 View commit details
    Browse the repository at this point in the history

  3. Fix names

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    f579417 View commit details
    Browse the repository at this point in the history

  4. Marge model before inference

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    9c813b2 View commit details
    Browse the repository at this point in the history

  5. Merge models before inference

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    a309f25 View commit details
    Browse the repository at this point in the history

  6. Fix bug

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    8ffa1c5 View commit details
    Browse the repository at this point in the history

  7. bf16 eval

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    d809598 View commit details
    Browse the repository at this point in the history

  8. Disable flash attention

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    7d58fa7 View commit details
    Browse the repository at this point in the history

  9. Prepare run 128

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    4c3e290 View commit details
    Browse the repository at this point in the history

  10. Set correct output path

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    d6d4b52 View commit details
    Browse the repository at this point in the history

  11. Eval 256

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    23d4fd0 View commit details
    Browse the repository at this point in the history

  12. Fix path

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    eba9e1f View commit details
    Browse the repository at this point in the history

  13. Use flash attention

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    ec4536a View commit details
    Browse the repository at this point in the history

  14. Eval 128

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    8cd3732 View commit details
    Browse the repository at this point in the history

  15. Reduce RAM to 100GB

    @ikergarcia1996
    ikergarcia1996 committed Oct 23, 2023
    Configuration menu
    Copy the full SHA
    ce090be View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2023

  1. hparam search

    @ikergarcia1996
    ikergarcia1996 committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    2626ccb View commit details
    Browse the repository at this point in the history

  2. Fix data path

    @ikergarcia1996
    ikergarcia1996 committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    33e5455 View commit details
    Browse the repository at this point in the history

  3. Add experiment

    @ikergarcia1996
    ikergarcia1996 committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    f616c06 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. Eval FT

    @ikergarcia1996
    ikergarcia1996 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    82c02bf View commit details
    Browse the repository at this point in the history

  2. eval

    @ikergarcia1996
    ikergarcia1996 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    670ea93 View commit details
    Browse the repository at this point in the history

  3. Remove ontonotes from eval (too slow for hparam search)

    @ikergarcia1996
    ikergarcia1996 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    beef851 View commit details
    Browse the repository at this point in the history

  4. Re run

    @ikergarcia1996
    ikergarcia1996 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    775bfe2 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. Attempt to fix crash

    @ikergarcia1996
    ikergarcia1996 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    f316017 View commit details
    Browse the repository at this point in the history

  2. Do not run large datasets

    @ikergarcia1996
    ikergarcia1996 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    570a5d1 View commit details
    Browse the repository at this point in the history

  3. Change paths

    @ikergarcia1996
    ikergarcia1996 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    8e0b0a7 View commit details
    Browse the repository at this point in the history

  4. Change params

    @ikergarcia1996
    ikergarcia1996 committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    ab69089 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2023

  1. Remove split size

    @ikergarcia1996
    ikergarcia1996 committed Oct 28, 2023
    Configuration menu
    Copy the full SHA
    3f85820 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2023

  1. Implement do not shuffle and disable paraphrase

    @ikergarcia1996
    ikergarcia1996 committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    8078269 View commit details
    Browse the repository at this point in the history

  2. Script to get seen and unseen labels

    @ikergarcia1996
    ikergarcia1996 committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    6e942be View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2023

  1. Implement loss from "text" and "all"

    @ikergarcia1996
    ikergarcia1996 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    46fcf5f View commit details
    Browse the repository at this point in the history

  2. Remove environment

    @ikergarcia1996
    ikergarcia1996 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    26cb61d View commit details
    Browse the repository at this point in the history

  3. Fix typo

    @ikergarcia1996
    ikergarcia1996 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    19ab3d0 View commit details
    Browse the repository at this point in the history

  4. Install flash attention in the node

    @ikergarcia1996
    ikergarcia1996 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    099cd60 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. Fix model

    @ikergarcia1996
    ikergarcia1996 committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    d4818e5 View commit details
    Browse the repository at this point in the history

  2. Set xlong partition

    @ikergarcia1996
    ikergarcia1996 committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    ee2d42b View commit details
    Browse the repository at this point in the history

  3. Set correct partition

    @ikergarcia1996
    ikergarcia1996 committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    f6f4c77 View commit details
    Browse the repository at this point in the history

  4. Fix output path

    @ikergarcia1996
    ikergarcia1996 committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    22fe23c View commit details
    Browse the repository at this point in the history

  5. Fix wandb

    @ikergarcia1996
    ikergarcia1996 committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    268b2c4 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. Change batch size

    @ikergarcia1996
    ikergarcia1996 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    11128b1 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Reduce batch size

    @ikergarcia1996
    ikergarcia1996 committed Nov 30, 2023
    Configuration menu
    Copy the full SHA
    f7e80f8 View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. DeepSeek

    @ikergarcia1996
    ikergarcia1996 committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    678005c View commit details
    Browse the repository at this point in the history

  2. Deepseek

    @ikergarcia1996
    ikergarcia1996 committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    ee1d7b6 View commit details
    Browse the repository at this point in the history

  3. Restore sh

    @ikergarcia1996
    ikergarcia1996 committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    944d8ab View commit details
    Browse the repository at this point in the history

  4. Fix run command

    @ikergarcia1996
    ikergarcia1996 committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    d0b7687 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. Fix eval

    @ikergarcia1996
    ikergarcia1996 committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    95339e3 View commit details
    Browse the repository at this point in the history

  2. Deepseek

    @ikergarcia1996
    ikergarcia1996 committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    46bac0d View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2023

  1. Increase GPU mem

    @ikergarcia1996
    ikergarcia1996 committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    070b22d View commit details
    Browse the repository at this point in the history

  2. Deepspeed

    @ikergarcia1996
    ikergarcia1996 committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    4137bf9 View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2023

  1. Fix eps string deepspeed

    @ikergarcia1996
    ikergarcia1996 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    9953ea5 View commit details
    Browse the repository at this point in the history

  2. Eval

    @ikergarcia1996
    ikergarcia1996 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    561fc46 View commit details
    Browse the repository at this point in the history

  3. Test new models

    @ikergarcia1996
    ikergarcia1996 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    6396526 View commit details
    Browse the repository at this point in the history

  4. Ensure 4 bit inference

    @ikergarcia1996
    ikergarcia1996 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    0d8b20c View commit details
    Browse the repository at this point in the history

  5. Reduce hw

    @ikergarcia1996
    ikergarcia1996 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    299733c View commit details
    Browse the repository at this point in the history

  6. Fix

    @ikergarcia1996
    ikergarcia1996 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    34ffd7f View commit details
    Browse the repository at this point in the history

  7. 8 gpus

    @ikergarcia1996
    ikergarcia1996 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    dc7e570 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. Fix optimizer

    @ikergarcia1996
    ikergarcia1996 committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    561bccc View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. Tune params

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    a82909e View commit details
    Browse the repository at this point in the history

  2. Reduce batch

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    65c53c3 View commit details
    Browse the repository at this point in the history

  3. Batch size 1

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    132d17c View commit details
    Browse the repository at this point in the history

  4. Eval

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    ff1c372 View commit details
    Browse the repository at this point in the history

  5. Verbosity

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    6bcd28f View commit details
    Browse the repository at this point in the history

  6. Verbosity

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    3fde6fb View commit details
    Browse the repository at this point in the history

  7. Verbosity

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    b6fca26 View commit details
    Browse the repository at this point in the history

  8. Deepspeed launcher

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    6a93b67 View commit details
    Browse the repository at this point in the history

  9. 4 bit quantization

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    9643469 View commit details
    Browse the repository at this point in the history

  10. Set low_cpu_mem_usage

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    b8eef08 View commit details
    Browse the repository at this point in the history

  11. Auto device_map

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    14adf1c View commit details
    Browse the repository at this point in the history

  12. Disable merge and upload

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    441a8c2 View commit details
    Browse the repository at this point in the history

  13. Auto device map

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    52beb8b View commit details
    Browse the repository at this point in the history

  14. Run without torchrun

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    3e5712f View commit details
    Browse the repository at this point in the history

  15. Auto devicemap

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    239d9a2 View commit details
    Browse the repository at this point in the history

  16. Attempt deepspeed

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    4b58264 View commit details
    Browse the repository at this point in the history

  17. Auto_devicemap=True

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    7f31a63 View commit details
    Browse the repository at this point in the history

  18. Quantization

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    34d09ee View commit details
    Browse the repository at this point in the history

  19. Set 1 batch size

    @ikergarcia1996
    ikergarcia1996 committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    3417370 View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Yi cosine scheduler

    @ikergarcia1996
    ikergarcia1996 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    781e5a1 View commit details
    Browse the repository at this point in the history

  2. No quantization

    @ikergarcia1996
    ikergarcia1996 committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    52e7cc6 View commit details
    Browse the repository at this point in the history

Commits on Dec 15, 2023

  1. Eval YI

    @ikergarcia1996
    ikergarcia1996 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    9ef641a View commit details
    Browse the repository at this point in the history

  2. Attempt to fix deepspeed inference with LoRA

    @ikergarcia1996
    ikergarcia1996 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    238ae32 View commit details
    Browse the repository at this point in the history

  3. Launch with accelerate

    @ikergarcia1996
    ikergarcia1996 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    06d084d View commit details
    Browse the repository at this point in the history

  4. Run without deepspeed

    @ikergarcia1996
    ikergarcia1996 committed Dec 15, 2023
    Configuration menu
    Copy the full SHA
    e74d3e7 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2024

  1. Update load model. Add MOE compatibility. Fix deprecated tests.

    @ikergarcia1996
    ikergarcia1996 committed Apr 24, 2024
    Configuration menu
    Copy the full SHA
    fb6ce83 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. Update paraphrase code

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    594bfe0 View commit details
    Browse the repository at this point in the history

  2. Fix bug

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    65a8584 View commit details
    Browse the repository at this point in the history

  3. mgpu

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    d23e739 View commit details
    Browse the repository at this point in the history

  4. Fix

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    d364556 View commit details
    Browse the repository at this point in the history

  5. Fix

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    5026aee View commit details
    Browse the repository at this point in the history

  6. Fix

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    a3fd29b View commit details
    Browse the repository at this point in the history

  7. Debug

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    9f38a6f View commit details
    Browse the repository at this point in the history

  8. Disable mgpu

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    a7e99c2 View commit details
    Browse the repository at this point in the history

  9. Remove labels if present

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    dadab39 View commit details
    Browse the repository at this point in the history

  10. Fix

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    200fa48 View commit details
    Browse the repository at this point in the history

  11. Deepspeed and fix output

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    3768bbd View commit details
    Browse the repository at this point in the history

  12. Fix

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    578d956 View commit details
    Browse the repository at this point in the history

  13. Fix

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    4775307 View commit details
    Browse the repository at this point in the history

  14. bf16

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    a41e4f0 View commit details
    Browse the repository at this point in the history

  15. Fix decoding

    @ikergarcia1996
    ikergarcia1996 committed May 23, 2024
    Configuration menu
    Copy the full SHA
    2db45b6 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. Add end to end

    @osainz59
    osainz59 committed May 24, 2024
    Configuration menu
    Copy the full SHA
    d3fc9d8 View commit details
    Browse the repository at this point in the history

  2. Make style

    @osainz59
    osainz59 committed May 24, 2024
    Configuration menu
    Copy the full SHA
    e1b5bdd View commit details
    Browse the repository at this point in the history

Commits on May 25, 2024

  1. add end2end configs

    @osainz59
    osainz59 committed May 25, 2024
    Configuration menu
    Copy the full SHA
    63b00e0 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. gitignore update

    @Neilus03
    Neilus03 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    8d1dde1 View commit details
    Browse the repository at this point in the history

  2. CHANGES IN DATA GENERATION, NEW LLAMA3 MODEL ADDITION AND GITIGNORE

    @Neilus03
    Neilus03 committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    cb243aa View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. added updates on run.py and configs for llama3.1

    @Neilus03
    Neilus03 committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    181aff7 View commit details
    Browse the repository at this point in the history

  2. updated run name

    @Neilus03
    Neilus03 committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    ef01604 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. changes in the bash and gitignore files

    @Neilus03
    Neilus03 committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    4983110 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. changes for pretraining on new dataset

    @Neilus03
    Neilus03 committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    2dbadb1 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. Finished sh

    @Neilus03
    Neilus03 committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    b4f5758 View commit details
    Browse the repository at this point in the history

  2. chore: Update GPU resource allocation for pretraining script

    @Neilus03
    Neilus03 committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    30e39bc View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. changes in the deepspeed config 2 -> 3 and added task_id mapping

    @Neilus03
    Neilus03 committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    cbadd48 View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. chore: Update GPU resource allocation for pretraining script

    @Neilus03
    Neilus03 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    d718178 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. chore: Update GPU resource allocation and batch sizes for pretraining… 

    … script
    @Neilus03
    Neilus03 committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    72060ab View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. Update GPU resource allocation and batch sizes for pretraining script

    @Neilus03
    Neilus03 committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    b901623 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. yaml and sh files edited for finetuning od pretrained model

    @Neilus03
    Neilus03 committed Sep 6, 2024
    Configuration menu
    Copy the full SHA
    4dd414a View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2024

  1. Update GPU resource allocation and batch sizes for pretraining script

    @Neilus03
    Neilus03 committed Sep 8, 2024
    Configuration menu
    Copy the full SHA
    3b095ce View commit details
    Browse the repository at this point in the history

  2. update

    @Neilus03
    Neilus03 committed Sep 8, 2024
    Configuration menu
    Copy the full SHA
    06de601 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Update GPU resource allocation and batch sizes for pretraining script

    @Neilus03
    Neilus03 committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    cc57a13 View commit details
    Browse the repository at this point in the history

  2. Update GPU resource allocation and batch sizes for pretraining script

    @Neilus03
    Neilus03 committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    7bca997 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2024

  1. chore: frequent logging, model to device, less gpus

    @Neilus03
    Neilus03 committed Sep 13, 2024
    Configuration menu
    Copy the full SHA
    47979d3 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. Refactor typing imports and annotations

    @Neilus03
    Neilus03 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    7eeaa1e View commit details
    Browse the repository at this point in the history

  2. CREATED NEW PRETRAINING STRATEGIES

    @Neilus03
    Neilus03 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    17b86ee View commit details
    Browse the repository at this point in the history

  3. DELETE TREE

    @Neilus03
    Neilus03 committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    4abc148 View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2024

  1. Update pretraining script for masking and curriculum strategies

    @Neilus03
    Neilus03 committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    70ac762 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. compute results to finetuning_normal_pretrained_vs_finetuning_masked_… 

    …pretrained.csv
    @Neilus03
    Neilus03 committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    7bbacaf View commit details
    Browse the repository at this point in the history

  2. Add bash script for finetuning with masking and negatives pretrained … 

    …models
    @Neilus03
    Neilus03 committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    e7cbd27 View commit details
    Browse the repository at this point in the history

  3. Update pretraining and finetuning config yamls for masking and negati… 

    …ves pretrained models
    @Neilus03
    Neilus03 committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    be209cc View commit details
    Browse the repository at this point in the history

  4. Update config.py for entity type masking and negatives probabilities

    @Neilus03
    Neilus03 committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    bee0286 View commit details
    Browse the repository at this point in the history

  5. Apply entity type masking with probability p to JSON entry, todo: neg… 

    …atives
    @Neilus03
    Neilus03 committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    5228b94 View commit details
    Browse the repository at this point in the history

  6. Apply entity type masking and negatives transformations to Collie dat… 

    …aset by creating a new Dataset CollieDatasetWithTransformations
    @Neilus03
    Neilus03 committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    c540296 View commit details
    Browse the repository at this point in the history

  7. Apply entity type masking and negatives transformations while training

    @Neilus03
    Neilus03 committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    206f269 View commit details
    Browse the repository at this point in the history

  8. deleting old unused files

    @Neilus03
    Neilus03 committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    162270d View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. NEGATIVES TO BE MODIFIED YET, NOT WORKING PROPERLY YET

    @Neilus03
    Neilus03 committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    2c6f53b View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. PRETRAINING & FINETUNING BASH SCRIPTS

    @Neilus03
    Neilus03 committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    b714a0f View commit details
    Browse the repository at this point in the history

  2. PRETRAINING AND FINETUNING CONFIG YAML FILES

    @Neilus03
    Neilus03 committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    bafc96d View commit details
    Browse the repository at this point in the history

  3. APPLY_NEGATIVES FUNCTION FINISHED AND WORKING

    @Neilus03
    Neilus03 committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    20719e4 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2024

  1. negatives loading on dataset.py, old without negatives in dataset_old

    @Neilus03
    Neilus03 committed Oct 12, 2024
    Configuration menu
    Copy the full SHA
    d87dbcc View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. sentences split correctes and unlabelled text split corrected

    @Neilus03
    Neilus03 committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    b70e38c View commit details
    Browse the repository at this point in the history

  2. data transformations for negatives fixed, created a function to turn … 

    …results into a table
    @Neilus03
    Neilus03 committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    3679a41 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. Fix data transformations for negatives and add table conversion function

    @Neilus03
    Neilus03 committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    1e57c54 View commit details
    Browse the repository at this point in the history

  2. Fix GPU allocation and memory usage in bash scripts for negatives pre… 

    …training
    @Neilus03
    Neilus03 committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    c889a59 View commit details
    Browse the repository at this point in the history

  3. created yaml configs for negatives pretraining and finetuning

    @Neilus03
    Neilus03 committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    7d5c69b View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2024

  1. Fix GPU allocation and memory usage in bash scripts for negatives and… 

    … curriculum pretraining
    @Neilus03
    Neilus03 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    4df4db4 View commit details
    Browse the repository at this point in the history

  2. Refactor dataset class selection and add num_workers argument 

    This commit refactors the dataset class selection in the `train_collie` function. If either `data_args.entity_type_masking_prob` or `data_args.negatives_prob` is set, the `CollieDatasetWithTransformations` class is used. Otherwise, the base `CollieDataset` class is used. Additionally, the `num_workers` argument is added to limit the number of workers to the minimum of the available CPU count and 16.
    @Neilus03
    Neilus03 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    07c11ec View commit details
    Browse the repository at this point in the history

  3. Refactor dataset class selection to accept the augmentations as a lis… 

    …t for curriculum learning possibility
    @Neilus03
    Neilus03 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    75786d1 View commit details
    Browse the repository at this point in the history

  4. curriculum finetuning configs

    @Neilus03
    Neilus03 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    13ddf4b View commit details
    Browse the repository at this point in the history

  5. curriculum pretraining configs

    @Neilus03
    Neilus03 committed Oct 21, 2024
    Configuration menu
    Copy the full SHA
    a569fc7 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. 30k pretraining and finetuning yamls and slurms

    @Neilus03
    Neilus03 committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    4da1650 View commit details
    Browse the repository at this point in the history

  2. avoid overfitting in pretraining by doing a single epoch

    @Neilus03
    Neilus03 committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    76b999e View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. less gpus

    @Neilus03
    Neilus03 committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    555a6e7 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2024

  1. 50k datapoint pretrainging and finetuning setups

    @Neilus03
    Neilus03 committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    7458b45 View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2024

  1. increased gpus on weekend

    @Neilus03
    Neilus03 committed Nov 16, 2024
    Configuration menu
    Copy the full SHA
    89df5c9 View commit details
    Browse the repository at this point in the history

  2. update finetuning configurations to include pretrained lora weights a… 

    …nd add new bash scripts for pretraining and finetuning
    @Neilus03
    Neilus03 committed Nov 16, 2024
    Configuration menu
    Copy the full SHA
    d0a3c43 View commit details
    Browse the repository at this point in the history

  3. update pretraining and finetuning configurations to reflect 30k curri… 

    …culum and add new bash scripts for 50k training
    @Neilus03
    Neilus03 committed Nov 16, 2024
    Configuration menu
    Copy the full SHA
    adebf1f View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2024

  1. add finetuning configuration for 50k pretrained model with updated tr… 

    …aining and evaluation settings
    @Neilus03
    Neilus03 committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    6fcf424 View commit details
    Browse the repository at this point in the history

  2. add new bash scripts for 50k pretraining and finetuning configuration… 

    …s with updated model settings
    @Neilus03
    Neilus03 committed Nov 17, 2024
    Configuration menu
    Copy the full SHA
    760fa50 View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. less gpus monday

    @Neilus03
    Neilus03 committed Nov 18, 2024
    Configuration menu
    Copy the full SHA
    c112730 View commit details
    Browse the repository at this point in the history