Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated script to better work with slurm #39

Open
wants to merge 47 commits into
base: main
Choose a base branch
from
Open

Updated script to better work with slurm #39

wants to merge 47 commits into from

Commits on Jun 24, 2022

  1. added new dataset

    knoriy committed Jun 24, 2022
    Configuration menu
    Copy the full SHA
    da2b7d9 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2022

  1. added CpVoST

    knoriy committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    b7f5044 View commit details
    Browse the repository at this point in the history

  2. removed duplidate dataset

    knoriy committed Jun 27, 2022
    Configuration menu
    Copy the full SHA
    014f519 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2022

  1. added to do list

    knoriy committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    50305e4 View commit details
    Browse the repository at this point in the history

  2. First commit

    knoriy committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    0d911c5 View commit details
    Browse the repository at this point in the history

  3. _

    knoriy committed Jun 29, 2022
    Configuration menu
    Copy the full SHA
    ef4dfcb View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2022

  1. Updated yaml to work with slurm

    knoriy committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    59c0f49 View commit details
    Browse the repository at this point in the history

  2. merged laion-main and fixed conflicts

    knoriy committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    4147b41 View commit details
    Browse the repository at this point in the history

  3. move yml file back to root as it is being used by other scripts outsi… 

    …de of data_preprocess
    knoriy committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    fb95086 View commit details
    Browse the repository at this point in the history

  4. _

    knoriy committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    0ac7933 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2022

  1. renamed file

    knoriy committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    6d6f929 View commit details
    Browse the repository at this point in the history

  2. removed unwanted text

    knoriy committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    eacade6 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2022

  1. _

    knoriy committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    43af9a7 View commit details
    Browse the repository at this point in the history

  2. First commit

    knoriy committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    a1576fe View commit details
    Browse the repository at this point in the history

  3. added clause to avoid duplicate calculation

    knoriy committed Jul 11, 2022
    Configuration menu
    Copy the full SHA
    4e34bd3 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2022

  1. Screipt to process CREMA-D

    knoriy committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    68ca8ff View commit details
    Browse the repository at this point in the history

  2. fixed issue where file were saved in wrong area

    knoriy committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    caf739a View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2022

  1. script to download appropreate tsv

    knoriy committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    890d63f View commit details
    Browse the repository at this point in the history

  2. Added caching of dfs to avoid recomputing the same sf multiple time i… 

    …f running on multiple nodes
    knoriy committed Jul 13, 2022
    Configuration menu
    Copy the full SHA
    1e8f87b View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2022

  1. fixed error where .json did not conform with audio-dataset

    knoriy committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    7bf8ede View commit details
    Browse the repository at this point in the history

  2. processing setup for en-X and x_en

    knoriy committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    a5941ba View commit details
    Browse the repository at this point in the history

  3. processing setup for en-X and x_en

    knoriy committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    2f9951e View commit details
    Browse the repository at this point in the history

  4. setup for eng to X

    knoriy committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    2c3036d View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2022

  1. Fixed json

    knoriy committed Aug 11, 2022
    Configuration menu
    Copy the full SHA
    fb5d9b7 View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2022

  1. modified to process given language

    knoriy committed Aug 15, 2022
    Configuration menu
    Copy the full SHA
    68e37c0 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2022

  1. Fixed bug when a trailing / is needed

    knoriy committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    8ca6509 View commit details
    Browse the repository at this point in the history

  2. processing code for EmoV-DB

    knoriy committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    a664c0d View commit details
    Browse the repository at this point in the history

  3. processing and cleaning old files

    knoriy committed Aug 16, 2022
    Configuration menu
    Copy the full SHA
    5323d86 View commit details
    Browse the repository at this point in the history

Commits on Aug 22, 2022

  1. Improved multiprocessing workload, processing improve from 3 it/s to … 

    …30000+ it/s
    knoriy committed Aug 22, 2022
    Configuration menu
    Copy the full SHA
    bc1fbad View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2022

  1. Merge branch 'LAION-AI:main' into main

    @knoriy
    knoriy authored Oct 7, 2022
    Configuration menu
    Copy the full SHA
    678106c View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2022

  1. Merge branch 'LAION-AI:main' into main

    @knoriy
    knoriy authored Oct 30, 2022
    Configuration menu
    Copy the full SHA
    6db9c08 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2022

  1. add: processor for cv11.0

    knoriy committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    c0b7bec View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:knoriy/audio-dataset

    knoriy committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    4d545dd View commit details
    Browse the repository at this point in the history

  3. add: script to process common voice

    knoriy committed Dec 5, 2022
    Configuration menu
    Copy the full SHA
    812e399 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'LAION-AI:main' into main

    @knoriy
    knoriy authored Dec 5, 2022
    Configuration menu
    Copy the full SHA
    00a7537 View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2023

  1. updated cmd

    knoriy committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    1c07775 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of github.com:knoriy/audio-dataset

    knoriy committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    ded62af View commit details
    Browse the repository at this point in the history

  3. automated extrators and updated text

    knoriy committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    850b120 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2023

  1. fix: fixed but laguage was not set correctly, being set as train,test… 

    … or valid
    knoriy committed Jan 16, 2023
    Configuration menu
    Copy the full SHA
    a7265cf View commit details
    Browse the repository at this point in the history

Commits on May 4, 2023

  1. Merge pull request #2 from LAION-AI/main 

    merge from laion
    @knoriy
    knoriy authored May 4, 2023
    Configuration menu
    Copy the full SHA
    0e3ca94 View commit details
    Browse the repository at this point in the history

  2. updated to include audioset descriptions

    knoriy committed May 4, 2023
    Configuration menu
    Copy the full SHA
    3eb72e7 View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2023

  1. backup

    knoriy committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    bb85a09 View commit details
    Browse the repository at this point in the history

  2. backup

    knoriy committed Jun 19, 2023
    Configuration menu
    Copy the full SHA
    0544617 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2023

  1. updated json file text to include metadata

    knoriy committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    e1e3643 View commit details
    Browse the repository at this point in the history

  2. Added meta

    knoriy committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    6045d5d View commit details
    Browse the repository at this point in the history

  3. upload to s3

    knoriy committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    a919011 View commit details
    Browse the repository at this point in the history

Commits on Jul 7, 2023

  1. added emns dataset

    @knoriy
    knoriy committed Jul 7, 2023
    Configuration menu
    Copy the full SHA
    3fc2359 View commit details
    Browse the repository at this point in the history