[NON-PY-SDK] pandas version issue with MXNet 1.6 environment

[ehsanmok]

Describe the bug
The mxnet_p36 env seems to enforce pandas==0.24 somehow which makes using awslabs/gluonts (pip installed from master) to fail (with and without upgrading pip) because GluonTS requires pandas~=1.0

ContextualVersionConflict: (pandas 0.24.2 (/home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages), Requirement.parse('pandas~=1.0'), {'gluonts'})

To reproduce
Run this in a notebook cell in mxnet_p36 env

%%bash
if [[ `python -m pip freeze | grep enum34` ]]; then
    python -m pip uninstall -y enum34
fi
python -m pip install --upgrade -q sagemaker
python -m pip install -q git+https://github.com/awslabs/gluon-ts@master

Also it seems pandas requirement was not specified anywhere for MXNet here.

System information
A description of your system. Please provide:

• SageMaker Python SDK version: 1.55.4
• Framework name (eg. PyTorch) or algorithm (eg. KMeans): MXNet
• Framework version: 1.6
• Python version: python3.6
• CPU or GPU: Both
• Custom Docker image (Y/N): N

[laurenyu]

sorry for the delayed response here - just to clarify, you're referring to the mxnet_p36 environment in a SageMaker Notebook Instance, right? (just want to make sure I route this to the correct team)

[ehsanmok]

Yes, that's right. I observed the package installation order causes random dependecies error. For example, on mxnet_p36 if I install python -m pip install -q altair==4.1 first then python -m pip install -q git+https://github.com/awslabs/gluon-ts@master and using gluonts results in the error above but if I reverse the order it seems fine!

I suspect there's something wrong with how pip handles stuff in notebook instances and the fact that notebook instances are still using pip v10.0 instead of the newest v20.0 may contribute to the issue as well.

[laurenyu]

gotcha, thanks for the clarification. I'm going to pass this along to the team that owns Notebook Instances to see if they have any insight. thanks for your patience!

[neelamgehlot]

Hi @ehsanmok ,

We have deployed the newer version of pandas. This change will be deployed to all regions in coming week.

In the meantime, you can update the version of pandas using conda update pandas in mxnet_p36 environment to unblock yourself.

Please re-open this issue if you still didn't get >1.0 version of pandas by end of next week.

Thanks,
Neelam