Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add initial support for RWKV v6 #174

Merged
merged 8 commits into from
Jul 2, 2024
Merged

Add initial support for RWKV v6 #174

merged 8 commits into from
Jul 2, 2024

Commits on Jun 21, 2024

  1. Initial support for RWKV v6. 

    sequence inferencing doesn't work correctly yet

Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
    @MollySophia
    MollySophia committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    458298f View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. Fix sequence mode for RWKV v6 

    Thanks to @cryscan

Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
    @MollySophia
    MollySophia committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    10d973a View commit details
    Browse the repository at this point in the history

  2. RWKV v6: Add lora merging 

    Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
    @MollySophia
    MollySophia committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    9d5797f View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. Add tiny-rwkv test for RWKV v6 

    The expected_difference_sum values needs to be determined,
after making sure FP32 logits are nearly the same.

Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
    @MollySophia
    MollySophia committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    2c14946 View commit details
    Browse the repository at this point in the history

  2. RWKV v6: Make outputs correct and update test values 

    Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
    @MollySophia
    MollySophia committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    edea0c2 View commit details
    Browse the repository at this point in the history

  3. Apply some code-format changes 

    Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
    @MollySophia
    MollySophia committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    2c8daac View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. tests: Use tiny-rwkv-6v0-3m 

    Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
    @MollySophia
    MollySophia committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    29ca869 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Converter: Fix a problem when tensor cannot be converted to numpy 

    Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
    @MollySophia
    MollySophia committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    3c4c01f View commit details
    Browse the repository at this point in the history