Does Sagemaker support deepspeed?

[mingtan888]

I am trying to run huggingface transformers with deepspeed in Sagemaker. But I cannot find a way to use deepspeed.
I tried from sagemaker.huggingface import HuggingFace with the following hyperparameters and command. But it doesn't work for deepspeed, although I installed the deepspeed packages in the sagemaker notebook. The error is shown below as well.

Seems the meta-data for deepspeed cannot be found.

# hyperparameters, which are passed into the training job
hyperparameters={
    'model_name_or_path':'roberta-large',
    'task_name': 'mnli',
    'per_device_train_batch_size': 16,
    'per_device_eval_batch_size': 16,
    'do_train': True,
    'do_eval': True,
    'do_predict': True,
    'num_train_epochs': 2,
    'output_dir':'/opt/ml/model',
    'max_steps': 500,
    **'deepspeed': './tests/deepspeed/ds_config_zero3.json'**
}

# configuration for running training on smdistributed Model Parallel
mpi_options = {
    "enabled" : True,
    "processes_per_host" : 8,
}
smp_options = {
    "enabled":True,
    "parameters": {
        "microbatches": 4,
        "placement_strategy": "spread",
        "pipeline": "interleaved",
        "optimize": "speed",
        "partitions": 4,
        "ddp": True,
    }
}

distribution={
    "smdistributed": {"modelparallel": smp_options},
    "mpi": mpi_options
}

# git configuration to download our fine-tuning script
git_config = {'repo': 'https://github.com/huggingface/transformers.git', 'branch': 'v4.6.1'}


# instance configurations
instance_type='ml.p3.16xlarge'
instance_count=1
volume_size=200

# metric definition to extract the results
metric_definitions=[
     {'Name': 'train_runtime', 'Regex':"train_runtime.*=\D*(.*?)$"},
     {'Name': 'train_samples_per_second', 'Regex': "train_samples_per_second.*=\D*(.*?)$"},
     {'Name': 'epoch', 'Regex': "epoch.*=\D*(.*?)$"},
     {'Name': 'f1', 'Regex': "f1.*=\D*(.*?)$"},
     {'Name': 'exact_match', 'Regex': "exact_match.*=\D*(.*?)$"}]
     
     
huggingface_estimator = HuggingFace(entry_point='run_glue.py',
                                    source_dir='./examples/pytorch/text-classification',
                                    git_config=git_config,
                                    metrics_definition=metric_definitions,
                                    instance_type=instance_type,
                                    instance_count=instance_count,
                                    volume_size=volume_size,
                                    role=role,
                                    transformers_version='4.6',
                                    pytorch_version='1.7',
                                    py_version='py36',
                                    distribution= distribution,
                                    hyperparameters = hyperparameters,
                                    debugger_hook_config=False)     
     

The Error is

[1,4]<stderr>:    mod_name, mod_spec, pkg_name, script_name)
[1,4]<stderr>:  File "/opt/conda/lib/python3.6/runpy.py", line 85, in _run_code
[1,4]<stderr>:    exec(code, run_globals)
[1,4]<stderr>:  File "run_glue.py", line 535, in <module>
[1,4]<stderr>:    main()
[1,4]<stderr>:    main()
[1,4]<stderr>:  File "run_glue.py", line 188, in main
[1,4]<stderr>:    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
[1,4]<stderr>:  File "/opt/conda/lib/python3.6/site-packages/transformers/hf_argparser.py", line 187, in parse_args_into_dataclasses
[1,4]<stderr>:    obj = dtype(**inputs)
[1,4]<stderr>:  File "<string>", line 69, in __init__
[1,4]<stderr>:  File "/opt/conda/lib/python3.6/site-packages/transformers/training_args.py", line 668, in __post_init__
[1,4]<stderr>:    self.deepspeed_config_hf = DeepSpeedConfigHF(self)
[1,4]<stderr>:  File "/opt/conda/lib/python3.6/site-packages/transformers/integrations.py", line 301, in __init__
[1,4]<stderr>:    dep_version_check("deepspeed")
[1,4]<stderr>:  File "/opt/conda/lib/python3.6/site-packages/transformers/dependency_versions_check.py", line 47, in dep_version_check
[1,4]<stderr>:    require_version(deps[pkg], hint)
[1,4]<stderr>:  File "/opt/conda/lib/python3.6/site-packages/transformers/utils/versions.py", line 108, in require_version
[1,4]<stderr>:    f"The '{requirement}' distribution was not found and is required by this application. {hint}"
[1,4]<stderr>:importlib_metadata.PackageNotFoundError: No package metadata was found for The 'deepspeed>=0.3.16' distribution was not found and is required by this application.