Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[PDSM] Medical Testcases for benchmarking #157

Merged
merged 98 commits into from
Jul 18, 2024
Merged

[PDSM] Medical Testcases for benchmarking #157

merged 98 commits into from
Jul 18, 2024

Commits on Apr 2, 2024

  1. change iterations

    melis committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    5a8749e View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. I love python

    @SturmCamper
    SturmCamper committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    4ecca83 View commit details
    Browse the repository at this point in the history

  2. - Local-Test-Remove in conftest.py 

    - Remove no_podcast.py cause not so wichtig
    @SturmCamper
    SturmCamper committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    b3b18a9 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2024

  1. Working project version

    @marlis-en
    marlis-en committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    d79cb67 View commit details
    Browse the repository at this point in the history

  2. First pytest not working

    @marlis-en
    marlis-en committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    813e2c1 View commit details
    Browse the repository at this point in the history

  3. 1 test geht

    @SeraLunatic
    SeraLunatic committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    ddd8bf7 View commit details
    Browse the repository at this point in the history

  4. test name

    @SeraLunatic
    SeraLunatic committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    41ddf2a View commit details
    Browse the repository at this point in the history

  5. second test

    @SeraLunatic
    SeraLunatic committed Apr 21, 2024
    Configuration menu
    Copy the full SHA
    29a62b8 View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. added physikum questions

    @SeraLunatic
    SeraLunatic committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    3a38516 View commit details
    Browse the repository at this point in the history

  2. regex suche

    @SeraLunatic
    SeraLunatic committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    fb7c431 View commit details
    Browse the repository at this point in the history

  3. lowercase wieder rein ups

    @SeraLunatic
    SeraLunatic committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    0f772de View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2024

  1. ADD Emergency Testcase

    Yasmin Tehranchian committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    fb7905d View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. CHANGE: regex and not regex in one runtest and csv

    @SeraLunatic
    SeraLunatic committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    827aad8 View commit details
    Browse the repository at this point in the history

  2. REFACTOR test names changed

    @SeraLunatic
    SeraLunatic committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    7b1d785 View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. ADDED new regex EEG-Questions

    @SeraLunatic
    SeraLunatic committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    3356c9e View commit details
    Browse the repository at this point in the history

  2. ADDED new yes_no EEG-Questions

    @SeraLunatic
    SeraLunatic committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    392da50 View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. Merge pull request #1 from mehizli/datascience_fragen 

    Datascience fragen
    @ytehran
    ytehran authored May 14, 2024
    Configuration menu
    Copy the full SHA
    a0ca30a View commit details
    Browse the repository at this point in the history

  2. Merge pull request #2 from mehizli/testcase_emergency 

    ADD Emergency Testcase
    @ytehran
    ytehran authored May 14, 2024
    Configuration menu
    Copy the full SHA
    9b4cf61 View commit details
    Browse the repository at this point in the history

  3. test

    melis committed May 14, 2024
    Configuration menu
    Copy the full SHA
    a3788b2 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'develop' into meli-test-file 

    # Conflicts:
#	benchmark/data/benchmark_data.yaml
    melis committed May 14, 2024
    Configuration menu
    Copy the full SHA
    8dcc567 View commit details
    Browse the repository at this point in the history

  5. indent fixes

    melis committed May 14, 2024
    Configuration menu
    Copy the full SHA
    8b1321d View commit details
    Browse the repository at this point in the history

  6. Merge pull request #3 from mehizli/meli-test-file 

    Meli test file
    @ytehran
    ytehran authored May 14, 2024
    Configuration menu
    Copy the full SHA
    84abb7d View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. FIXED Encoding Bug

    @SeraLunatic
    SeraLunatic committed May 15, 2024
    Configuration menu
    Copy the full SHA
    4caf83a View commit details
    Browse the repository at this point in the history

  2. Merge pull request #4 from mehizli/bugfix/data_not_found 

    FIXED Encoding Bug
    @SeraLunatic
    SeraLunatic authored May 15, 2024
    Configuration menu
    Copy the full SHA
    886c336 View commit details
    Browse the repository at this point in the history

  3. Added new testcases for mental diseases

    @marlis-en
    marlis-en committed May 15, 2024
    Configuration menu
    Copy the full SHA
    1768961 View commit details
    Browse the repository at this point in the history

  4. test cases fixes

    melis committed May 15, 2024
    Configuration menu
    Copy the full SHA
    372e3c8 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'develop' into meli-test-file 

    # Conflicts:
#	benchmark/data/benchmark_data.yaml
    melis committed May 15, 2024
    Configuration menu
    Copy the full SHA
    d731dba View commit details
    Browse the repository at this point in the history

  6. test cases indent fixes

    melis committed May 15, 2024
    Configuration menu
    Copy the full SHA
    e5be05f View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. added englisch translations

    melis committed May 20, 2024
    Configuration menu
    Copy the full SHA
    20183cb View commit details
    Browse the repository at this point in the history

  2. test for oncology

    melis committed May 20, 2024
    Configuration menu
    Copy the full SHA
    b177a7d View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. ADD translation for physikum

    @SeraLunatic
    SeraLunatic committed May 22, 2024
    Configuration menu
    Copy the full SHA
    3dd8930 View commit details
    Browse the repository at this point in the history

  2. ADD translation EEG

    @SeraLunatic
    SeraLunatic committed May 22, 2024
    Configuration menu
    Copy the full SHA
    5e2d180 View commit details
    Browse the repository at this point in the history

  3. REFACTORED and ADDED questions for mental diseases

    @marlis-en
    marlis-en committed May 22, 2024
    Configuration menu
    Copy the full SHA
    a53a8be View commit details
    Browse the repository at this point in the history

  4. WIP wrong_asnwer csv

    @SeraLunatic
    SeraLunatic committed May 22, 2024
    Configuration menu
    Copy the full SHA
    0a4b75a View commit details
    Browse the repository at this point in the history

  5. BUGFIX eeg_answer

    @SeraLunatic
    SeraLunatic committed May 22, 2024
    Configuration menu
    Copy the full SHA
    d3c6fce View commit details
    Browse the repository at this point in the history

  6. ADD writing csv with expected, wrong and failure_group

    @SeraLunatic
    SeraLunatic committed May 22, 2024
    Configuration menu
    Copy the full SHA
    c853b72 View commit details
    Browse the repository at this point in the history

  7. ADD regex failure_groups

    @SeraLunatic
    SeraLunatic committed May 22, 2024
    Configuration menu
    Copy the full SHA
    f3a1597 View commit details
    Browse the repository at this point in the history

  8. ADD regex failure_groups and better synonym tracker

    @SeraLunatic
    SeraLunatic committed May 22, 2024
    Configuration menu
    Copy the full SHA
    4d67b3c View commit details
    Browse the repository at this point in the history

  9. Merge pull request #5 from mehizli/meli-test-file 

    Meli test file
    @ytehran
    ytehran authored May 22, 2024
    Configuration menu
    Copy the full SHA
    64e27d4 View commit details
    Browse the repository at this point in the history

  10. Merge pull request #6 from mehizli/translate_quest 

    Translate quest
    @ytehran
    ytehran authored May 22, 2024
    Configuration menu
    Copy the full SHA
    7ccd4f4 View commit details
    Browse the repository at this point in the history

  11. Merge pull request #7 from mehizli/testcase_mental_diseases 

    REFACTORED and ADDED questions for mental diseases
    @ytehran
    ytehran authored May 22, 2024
    Configuration menu
    Copy the full SHA
    83b4905 View commit details
    Browse the repository at this point in the history

  12. Merge pull request #8 from mehizli/develop 

    Aktueller Stand
    @ytehran
    ytehran authored May 22, 2024
    Configuration menu
    Copy the full SHA
    ad7a0da View commit details
    Browse the repository at this point in the history

  13. Merge branch 'wrong_answers' into develop 

    # Conflicts:
#	benchmark/data/benchmark_data.yaml
    @SeraLunatic
    SeraLunatic committed May 22, 2024
    Configuration menu
    Copy the full SHA
    b48d03f View commit details
    Browse the repository at this point in the history

  14. Merge pull request #9 from mehizli/develop 

    Develop
    @ytehran
    ytehran authored May 22, 2024
    Configuration menu
    Copy the full SHA
    861282a View commit details
    Browse the repository at this point in the history

  15. Translate the emergency cases and updating to the new schema

    Yasmin Tehranchian committed May 22, 2024
    Configuration menu
    Copy the full SHA
    2b4f6ad View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. New cardiology cases in German

    Yasmin Tehranchian committed May 24, 2024
    Configuration menu
    Copy the full SHA
    9f4e039 View commit details
    Browse the repository at this point in the history

  2. Case translated into English

    Yasmin Tehranchian committed May 24, 2024
    Configuration menu
    Copy the full SHA
    f871db0 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #10 from mehizli/meli-test-file 

    test for oncology
    @ytehran
    ytehran authored May 24, 2024
    Configuration menu
    Copy the full SHA
    74b821c View commit details
    Browse the repository at this point in the history

  4. Merge branch 'develop' into cardio_update 

    # Conflicts:
#	benchmark/data/benchmark_data.yaml
    Yasmin Tehranchian committed May 24, 2024
    Configuration menu
    Copy the full SHA
    eb73efb View commit details
    Browse the repository at this point in the history

  5. Refactor and clean code

    Yasmin Tehranchian committed May 24, 2024
    Configuration menu
    Copy the full SHA
    4a4fd38 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #11 from mehizli/cardio_update 

    Cardio update
    @ytehran
    ytehran authored May 24, 2024
    Configuration menu
    Copy the full SHA
    4b53bb0 View commit details
    Browse the repository at this point in the history

  7. Merge pull request #12 from mehizli/develop 

    Develop
    @ytehran
    ytehran authored May 24, 2024
    Configuration menu
    Copy the full SHA
    d4c433d View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. ADDED new testcase dermatology

    @marlis-en
    marlis-en committed May 27, 2024
    Configuration menu
    Copy the full SHA
    72c7429 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2024

  1. formatting

    @slobentanzer
    slobentanzer committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    be9cdc9 View commit details
    Browse the repository at this point in the history

  2. revert doubly run tests

    @slobentanzer
    slobentanzer committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    14490f1 View commit details
    Browse the repository at this point in the history

  3. revert non-skipping

    @slobentanzer
    slobentanzer committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    5c10b8b View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. comment out models for test purposes

    @slobentanzer
    slobentanzer committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    2714b9a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into pr/ytehran/157

    @slobentanzer
    slobentanzer committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    49e4e41 View commit details
    Browse the repository at this point in the history

  3. pre-commit

    @slobentanzer
    slobentanzer committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    018191e View commit details
    Browse the repository at this point in the history

  4. correct function name

    @slobentanzer
    slobentanzer committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    5328bd9 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into pr/ytehran/157

    @slobentanzer
    slobentanzer committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    af82910 View commit details
    Browse the repository at this point in the history

  6. change nomenclature: wrong_result -> failure_mode 

    we record `actual_answer`s in cases where the response is not exactly equal to the expected response
    @slobentanzer
    slobentanzer committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    c137e4c View commit details
    Browse the repository at this point in the history

  7. record single scores, calculate standard deviation 

    scores concatenated via semicolon and split in `hooks.py`
SD calculation also only in `hooks.py` after splitting
    @slobentanzer
    slobentanzer committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    b1125cb View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. delete venv/.env in gitignore

    melis committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    fffadab View commit details
    Browse the repository at this point in the history

  2. delete test results

    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    8477af7 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' of https://github.com/mehizli/biochatter_pdsm int… 

    …o pr/ytehran/157
    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    4caed92 View commit details
    Browse the repository at this point in the history

  4. one iteration for dev

    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    5cb9e52 View commit details
    Browse the repository at this point in the history

  5. bring back text extraction results

    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    b3e2908 View commit details
    Browse the repository at this point in the history

  6. rename correctness to medical_exam, more specific

    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    be8562e View commit details
    Browse the repository at this point in the history

  7. run once on openhermes

    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    d80efe6 View commit details
    Browse the repository at this point in the history

  8. replace case separator (colon) 

    to distinguish between multi-word ('one_word') and categories
    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    b24b8d3 View commit details
    Browse the repository at this point in the history

  9. reset results

    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    bf3808f View commit details
    Browse the repository at this point in the history

  10. run once on representative models

    @slobentanzer
    slobentanzer committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    6ce37f0 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. REFACTOR testcases mental diseases and dermatology

    @marlis-en
    marlis-en committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    8aeaaae View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2024

  1. little fixes + einheitliches format der fragen

    melis committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    b73944e View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2024

  1. Refine the categorize_failures method

    Yasmin Tehranchian committed Jun 16, 2024
    Configuration menu
    Copy the full SHA
    c57d0ee View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/develop' into develop

    Yasmin Tehranchian committed Jun 16, 2024
    Configuration menu
    Copy the full SHA
    f5984db View commit details
    Browse the repository at this point in the history

  3. Merge branch 'develop' into promt-optimization 

    # Conflicts:
#	benchmark/benchmark_utils.py
#	benchmark/data/benchmark_data.yaml
    Yasmin Tehranchian committed Jun 16, 2024
    Configuration menu
    Copy the full SHA
    0651824 View commit details
    Browse the repository at this point in the history

  4. Refactor to part single and multiple choice

    Yasmin Tehranchian committed Jun 16, 2024
    Configuration menu
    Copy the full SHA
    dedc67f View commit details
    Browse the repository at this point in the history

  5. Merge pull request #14 from mehizli/promt-optimization 

    Promt optimization
    @ytehran
    ytehran authored Jun 16, 2024
    Configuration menu
    Copy the full SHA
    6ff88de View commit details
    Browse the repository at this point in the history

  6. Merge pull request #15 from mehizli/develop 

    Develop
    @ytehran
    ytehran authored Jun 16, 2024
    Configuration menu
    Copy the full SHA
    37d0916 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. first batch of LLMs run on med exam

    @slobentanzer
    slobentanzer committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    66293a9 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. another batch of models run

    @slobentanzer
    slobentanzer committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    a1f4807 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. another batch of open models run

    @slobentanzer
    slobentanzer committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    ac5845f View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. + j-notebook for analysis and graphs

    @SturmCamper
    SturmCamper committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    112bd8d View commit details
    Browse the repository at this point in the history

  2. Remove due to Deprecation (Just throwing errors while benchmarking)

    @SturmCamper
    SturmCamper committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    e9bcc7b View commit details
    Browse the repository at this point in the history

  3. J-Notebook, for simple Graph generation

    @SturmCamper
    SturmCamper committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    f8393e8 View commit details
    Browse the repository at this point in the history

  4. Added documentation of functions

    @marlis-en
    marlis-en committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    99260b6 View commit details
    Browse the repository at this point in the history

  5. Documentation of the data analysis

    @marlis-en
    marlis-en committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    770802c View commit details
    Browse the repository at this point in the history

  6. Added Lang/cat Graph

    @SturmCamper
    SturmCamper committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    2cb47c9 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'stats_graphs' of https://github.com/mehizli/biochatter_… 

    …pdsm into stats_graphs
    @SturmCamper
    SturmCamper committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    70f7cc8 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. Merge branch 'main' into main

    @slobentanzer
    slobentanzer authored Jul 2, 2024
    Configuration menu
    Copy the full SHA
    f40cae7 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into main

    @slobentanzer
    slobentanzer authored Jul 2, 2024
    Configuration menu
    Copy the full SHA
    ee0482b View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. Merge pull request #16 from mehizli/develop 

    Added documentation of functions
    @ytehran
    ytehran authored Jul 15, 2024
    Configuration menu
    Copy the full SHA
    c1490eb View commit details
    Browse the repository at this point in the history

  2. Changes to fix PR

    Yasmin Tehranchian committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    40086de View commit details
    Browse the repository at this point in the history

  3. Fixed the calc of the std

    sturmcmaper committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    c7a4695 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'stats_graphs' of github.com:mehizli/biochatter_pdsm int… 

    …o stats_graphs
    sturmcmaper committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    e753aec View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2024

  1. Merge pull request #18 from mehizli/stats_graphs 

    Stats graphs
    @ytehran
    ytehran authored Jul 16, 2024
    Configuration menu
    Copy the full SHA
    ad2070d View commit details
    Browse the repository at this point in the history