Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add feature to download google drive datasets #138

Merged
merged 37 commits into from
Aug 5, 2021
Merged

Add feature to download google drive datasets #138

merged 37 commits into from
Aug 5, 2021

Commits on Mar 25, 2021

  1. Add feature to download google drive datasets 

    Issue KwanLab#110
    @ajlail98
    ajlail98 committed Mar 25, 2021
    Configuration menu
    Copy the full SHA
    237d4a9 View commit details
    Browse the repository at this point in the history

  2. Add gdown to requirements.txt

    @ajlail98
    ajlail98 committed Mar 25, 2021
    Configuration menu
    Copy the full SHA
    c6fd46d View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2021

  1. 🎨 pulling from upstream dev

    @ajlail98
    ajlail98 committed Apr 29, 2021
    Configuration menu
    Copy the full SHA
    751bd7c View commit details
    Browse the repository at this point in the history

Commits on May 25, 2021

  1. 🎨 Formatted script according to template, renamed variables

    @ajlail98
    ajlail98 committed May 25, 2021
    Configuration menu
    Copy the full SHA
    b65d14e View commit details
    Browse the repository at this point in the history

  2. 🎨 Changed permissions

    @ajlail98
    ajlail98 committed May 25, 2021
    Configuration menu
    Copy the full SHA
    d7afab6 View commit details
    Browse the repository at this point in the history

  3. 🎨 Added unique filenames for each file size

    @ajlail98
    ajlail98 committed May 25, 2021
    Configuration menu
    Copy the full SHA
    0f35181 View commit details
    Browse the repository at this point in the history

  4. 🎨 Moved to external folder

    @ajlail98
    ajlail98 committed May 25, 2021
    Configuration menu
    Copy the full SHA
    fbb6c73 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2021

  1. Moved script to validation and renamed

    @ajlail98
    ajlail98 committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    72f0c99 View commit details
    Browse the repository at this point in the history

  2. Rename function and add type hints

    @ajlail98
    ajlail98 committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    bd92f5e View commit details
    Browse the repository at this point in the history

  3. Add file containing fileIDs to reference

    @ajlail98
    ajlail98 committed Jun 4, 2021
    Configuration menu
    Copy the full SHA
    ad70968 View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2021

  1. Add user input options for files/folders

    @ajlail98
    ajlail98 committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    b7df5f5 View commit details
    Browse the repository at this point in the history

  2. Reformat with black

    @ajlail98
    ajlail98 committed Jun 9, 2021
    Configuration menu
    Copy the full SHA
    0abe3a6 View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2021

  1. Change targets variable name

    @ajlail98
    ajlail98 committed Jun 10, 2021
    Configuration menu
    Copy the full SHA
    df63e97 View commit details
    Browse the repository at this point in the history

  2. Change "folder" to "dataset"

    @ajlail98
    ajlail98 committed Jun 10, 2021
    Configuration menu
    Copy the full SHA
    79484a5 View commit details
    Browse the repository at this point in the history

  3. Update column names

    @ajlail98
    ajlail98 committed Jun 10, 2021
    Configuration menu
    Copy the full SHA
    662d5bf View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2021

  1. Condense logic into one function

    @ajlail98
    ajlail98 committed Jun 11, 2021
    Configuration menu
    Copy the full SHA
    7678155 View commit details
    Browse the repository at this point in the history

  2. Change logic to input multiple files and multiple output dirs

    @ajlail98
    ajlail98 committed Jun 11, 2021
    Configuration menu
    Copy the full SHA
    3ffd397 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2021

  1. Add logger warnings

    @ajlail98
    ajlail98 committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    46eafc2 View commit details
    Browse the repository at this point in the history

  2. Add datasets.py info to setup.py

    @ajlail98
    ajlail98 committed Jun 15, 2021
    Configuration menu
    Copy the full SHA
    d21f825 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2021

  1. Change internet_is_connected into an import

    @ajlail98
    ajlail98 committed Jun 24, 2021
    Configuration menu
    Copy the full SHA
    54d151d View commit details
    Browse the repository at this point in the history

  2. Add internet connection checker and error message

    @ajlail98
    ajlail98 committed Jun 24, 2021
    Configuration menu
    Copy the full SHA
    3dd9e63 View commit details
    Browse the repository at this point in the history

Commits on Jul 13, 2021

  1. Directory structure to organize downloads

    @ajlail98
    ajlail98 committed Jul 13, 2021
    Configuration menu
    Copy the full SHA
    2a45ab2 View commit details
    Browse the repository at this point in the history

  2. Change variable names and clean up extra bits

    @ajlail98
    ajlail98 committed Jul 13, 2021
    Configuration menu
    Copy the full SHA
    b7c2048 View commit details
    Browse the repository at this point in the history

  3. Add __init__.py to validation

    @ajlail98
    ajlail98 committed Jul 13, 2021
    Configuration menu
    Copy the full SHA
    9a932d5 View commit details
    Browse the repository at this point in the history

  4. Add error for non-existent dir_path

    @ajlail98
    ajlail98 committed Jul 13, 2021
    Configuration menu
    Copy the full SHA
    98e356b View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2021

  1. Add detail to internet_is_connected failure

    @ajlail98
    ajlail98 committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    0d1274b View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2021

  1. Added NotImplementedError

    @ajlail98
    ajlail98 committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    7af3c95 View commit details
    Browse the repository at this point in the history

  2. Only read csv once

    @ajlail98
    ajlail98 committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    df317b0 View commit details
    Browse the repository at this point in the history

  3. Change strategy for filtering df

    @ajlail98
    ajlail98 committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    85c9387 View commit details
    Browse the repository at this point in the history

  4. Using df.loc to retrieve file_id

    @ajlail98
    ajlail98 committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    12afe4b View commit details
    Browse the repository at this point in the history

  5. Argparse and var name refinements

    @ajlail98
    ajlail98 committed Jul 16, 2021
    Configuration menu
    Copy the full SHA
    e7da939 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2021

  1. Add ability to ping custom IP

    @ajlail98
    ajlail98 committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    dceb0f5 View commit details
    Browse the repository at this point in the history

  2. Reformatting

    @ajlail98
    ajlail98 committed Jul 20, 2021
    Configuration menu
    Copy the full SHA
    622d934 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2021

  1. Hardcode fileID csv hosted on google drive

    @ajlail98
    ajlail98 committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    ac89c06 View commit details
    Browse the repository at this point in the history

  2. Reformatting

    @ajlail98
    ajlail98 committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    af931bb View commit details
    Browse the repository at this point in the history

  3. Remove gdown_fileIDs.csv

    @ajlail98
    ajlail98 committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    33f75e1 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2021

  1. Add verbose error message and dockerfile entrypoint

    @ajlail98
    ajlail98 committed Jul 30, 2021
    Configuration menu
    Copy the full SHA
    7fdf590 View commit details
    Browse the repository at this point in the history