Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP Multimodal support #1930

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open

WIP Multimodal support #1930

wants to merge 20 commits into from

Commits on Sep 24, 2024

  1. fixed small typo

    Sunny authored and sunny committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    424c068 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. wip for pixtral/multimodal integration

    sunny committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    7ee1cc1 View commit details
    Browse the repository at this point in the history

  2. WIP on pixtral/multimodal integration

    sunny committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    53c9744 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. added configuration argument support for multimodal

    sunny committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    a7cf2ad View commit details
    Browse the repository at this point in the history

  2. WIP on generalizing load_model for multimodal support

    sunny committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    b19ca47 View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. modified models.py and train.py for better integration of processor /… 

    … tolenizer
    sunny committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    15423a4 View commit details
    Browse the repository at this point in the history

  2. attempt at generalizing trainer builder for multimodal

    sunny committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    6a7744d View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2024

  1. fixed some format

    sunny committed Sep 28, 2024
    Configuration menu
    Copy the full SHA
    81dffce View commit details
    Browse the repository at this point in the history

  2. fixed code formatting

    sunny committed Sep 28, 2024
    Configuration menu
    Copy the full SHA
    dd1ec8e View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. fixed format

    sunny committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    4a10519 View commit details
    Browse the repository at this point in the history

  2. fixed format

    sunny committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    518e824 View commit details
    Browse the repository at this point in the history

  3. processor support for dataset/data collator

    sunny committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    f242c25 View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. progress towards multimodal integration

    sunny committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    cb31826 View commit details
    Browse the repository at this point in the history

  2. test

    sunny committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    0e511c3 View commit details
    Browse the repository at this point in the history

  3. progress towards multimodal support

    sunny committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    d5b3e0e View commit details
    Browse the repository at this point in the history

  4. conform to coding practice

    sunny committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    84ed3f5 View commit details
    Browse the repository at this point in the history

  5. conform to good coding practice

    sunny committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    c9b3def View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2024

  1. modified collators for multimodal

    sunny committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    93b33c2 View commit details
    Browse the repository at this point in the history

  2. modified collators for multimodal

    sunny committed Oct 2, 2024
    Configuration menu
    Copy the full SHA
    3ac5c1e View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. added chat template for llama3_2_vision

    sunny committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    0515d13 View commit details
    Browse the repository at this point in the history