Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix numpy 1.20+ deprecation warnings #42929

Merged

Conversation

zlsh80826
Copy link
Collaborator

PR types

Bug fixes

PR changes

Others

Describe

np.bool, np.int, np.float, np.complex, np.object, np.str, np.long, np.unicode are deprecated. There are tons of warnings during Paddle runs, those warnings are very distracting. This PR makes following changes

@paddle-bot-old
Copy link

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot-old paddle-bot-old bot added contributor External developers status: proposed labels May 23, 2022
@zlsh80826 zlsh80826 force-pushed the zlsh-fix-numpy-deprecation-warning branch from 2be10d5 to 17e3efb Compare May 23, 2022 06:56
@zlsh80826 zlsh80826 force-pushed the zlsh-fix-numpy-deprecation-warning branch from 17e3efb to d3b6321 Compare May 23, 2022 08:02
@zlsh80826 zlsh80826 force-pushed the zlsh-fix-numpy-deprecation-warning branch from d3b6321 to b482948 Compare May 23, 2022 08:55
@qili93 qili93 requested a review from ZHUI May 26, 2022 04:07
@zlsh80826 zlsh80826 force-pushed the zlsh-fix-numpy-deprecation-warning branch from b482948 to eb4d04d Compare May 27, 2022 05:48
@zlsh80826
Copy link
Collaborator Author

@qili93 There are many of CI failures after changing np.float to np.float32. It seems the variance should be np.float64, could you check it again?

@qili93
Copy link
Contributor

qili93 commented May 27, 2022

Got it, I will double confirm the failure CI test cases.

@qili93
Copy link
Contributor

qili93 commented Jun 1, 2022

hi @zlsh80826

Double confirmed Paddle code, the root cause is :

  1. the type: <class 'numpy.ndarray'> coverted to std::vector<int> by protobuf.cc https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/pybind/protobuf.cc#L289
  2. the attribute.h always got failure when it try to convert std::vector<int> to std::vector<float> https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/framework/attribute.h#L47

And the reason np.float64 works is : std::vector<double> defined how to convert int array to double https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/framework/attribute.h#L184

So propose to keep all the numpy array data type to np.float64 in python code, i.e. change the box_coder_op / density_prior_box_op / prior_box_op back to np.float64.

I have tried to add a similar function to handle std::vector<float> as std::vector<double> does, while it will result in accuracy loss. As described in the PR here #30126.

Apologize for my previous comment is not correct.

Thanks!
Qi

@paddle-bot-old
Copy link

paddle-bot-old bot commented Jun 4, 2022

Sorry to inform you that eb4d04d's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

@zlsh80826 zlsh80826 force-pushed the zlsh-fix-numpy-deprecation-warning branch from eb4d04d to aa3d10d Compare June 7, 2022 09:56
@qili93
Copy link
Contributor

qili93 commented Jun 7, 2022

hi, @zlsh80826

Seems some code style check failed in CI, propose to use this docker image to run git commit command, it will use clang-format installed in docker to format the code style automatically.

nvidia-docker run -it --name paddle-dev -v `pwd`:/workspace \
  --network=host --shm-size=128G --workdir=/workspace \
  --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
  registry.baidubce.com/paddlepaddle/paddle:latest-dev-cuda11.2-cudnn8-gcc82 /bin/bash

# install pre-commit inside container
pip3 install pre-commit

# then clang-format will format the code style automatically
git commit -m "......"

@zlsh80826 zlsh80826 force-pushed the zlsh-fix-numpy-deprecation-warning branch from 34ea207 to 29e8d95 Compare June 8, 2022 03:34
@zlsh80826
Copy link
Collaborator Author

Hi @qili93,
I already rerun the formatter. Could you help me to rerun the PR-CI-Coverage? It seems there is a network issue.

Copy link
Contributor

@qili93 qili93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qili93 qili93 requested a review from XiaoguangHu01 June 9, 2022 12:48
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qili93 qili93 merged commit 90cf229 into PaddlePaddle:develop Jun 14, 2022
qili93 pushed a commit to qili93/Paddle that referenced this pull request Jun 14, 2022
* Replace np.bool/np.bool8 with np.bool_

* Replace np.object with np.object_

* Replace np.complex with np.complex128

* Replace np.float with np.float64

* Replace np.int with np.int_

* Rerun pre-commit for newer pre-commit configuration

* Use builtin bool instead of np.bool_ based on the context
XiaoguangHu01 pushed a commit that referenced this pull request Jun 16, 2022
* Fix numpy 1.20+ deprecation warnings (#42929)

* Replace np.bool/np.bool8 with np.bool_

* Replace np.object with np.object_

* Replace np.complex with np.complex128

* Replace np.float with np.float64

* Replace np.int with np.int_

* Rerun pre-commit for newer pre-commit configuration

* Use builtin bool instead of np.bool_ based on the context

* fix mode dtype

Co-authored-by: zlsh80826 <rewang@nvidia.com>
@paddle-bot-old paddle-bot-old bot removed the contributor External developers label Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants