Adjust local dataset defaults (#156)

* Add default to download script and adjust yamls

Co-authored-by: dblalock <davis@mosaicml.com>

.gitignore

@@ -1,5 +1,6 @@
 # Data
 my-copy-c4/
+examples/deeplab/ade20k
 *.jsonl*
 
 # WandB

examples/deeplab/README.md

@@ -41,11 +41,11 @@ Now that you have explored the code, let's jump into the prerequisites for train
 
 ## Prepare your data
 
-This benchmark assumes that [ADE20k Dataset](https://groups.csail.mit.edu/vision/datasets/ADE20K/) is already stored on your local machine or stored in an S3 bucket after being processed into a streaming dataset. ADE20K can be downloaded by running:
+This benchmark assumes that [ADE20k Dataset](https://groups.csail.mit.edu/vision/datasets/ADE20K/) is already stored on your local machine or stored in an S3 bucket after being processed into a streaming dataset. ADE20K can be downloaded by running the command below. This takes up about 1GB of storage and will default to storing the dataset in `./ade20k`.
 
 ```bash
 # download ADE20k to specified local directory
-python download_ade20k.py path/to/data
+python download_ade20k.py
 ```
 
 To convert ADE20k to a [streaming format](https://github.com/mosaicml/streaming) for efficient training from an object store like S3, use [this script](https://github.com/mosaicml/streaming/blob/main/streaming/vision/convert/ade20k.py).

examples/deeplab/download_ade20k.py

@@ -13,7 +13,10 @@
 
 parser = argparse.ArgumentParser()
 
-parser.add_argument('path', help='ADE20k Download directory.', type=str)
+parser.add_argument('--path',
+                    help='ADE20k Download directory.',
+                    type=str,
+                    default='./ade20k')
 
 args = parser.parse_args()
 

examples/deeplab/yamls/deeplabv3.yaml

@@ -15,9 +15,9 @@ model:
 
 # Training Dataset Parameters
 train_dataset:
-  is_streaming: true                 # If true, use streaming dataset
-  path: s3://my-bucket/ade20k        # Path to S3 bucket if streaming, otherwise path to local data directory
-  local: /tmp/mds-cache/mds-ade20k/  # Local cache when streaming data
+  is_streaming: false                # If true, use streaming dataset
+  path: ./ade20k/                    # Path to S3 bucket if streaming, otherwise path to local data directory
+  local: null                        # Local cache when streaming data
   base_size: 512                     # Initial size of the image and target before other augmentations
   min_resize_scale: 0.5              # The minimum value the samples can be rescaled
   max_resize_scale: 2.0              # The maximum value the samples can be rescaled
@@ -27,9 +27,9 @@ train_dataset:
 
 # Validation Dataset Parameters
 eval_dataset:
-  is_streaming: true                   # If true, use streaming dataset
-  path: s3://my-bucket/ade20k          # Path to S3 bucket if streaming, otherwise path to local data directory
-  local: /tmp/mds-cache/mds-ade20k/    # Local cache when streaming data
+  is_streaming: false                  # If true, use streaming dataset
+  path: ./ade20k/                      # Path to S3 bucket if streaming, otherwise path to local data directory
+  local: null                          # Local cache when streaming data
   base_size: 512                       # Initial size of the image and target before other augmentations
   min_resize_scale: 0.5                # The minimum value the samples can be rescaled
   max_resize_scale: 2.0                # The maximum value the samples can be rescaled

examples/deeplab/yamls/mcloud_run.yaml

@@ -10,7 +10,7 @@ integrations:
 command: |
   cd examples/examples/deeplab
   composer main.py /mnt/config/parameters.yaml
-# Configuration copied from baseline.yaml
+# Configuration similar to baseline.yaml
 parameters:
   run_name: deeplabv3_ade20k # Name of the  training run used for checkpointing and other logging
   is_train: true             # Trains the model if true, otherwise runs evaluation