Evaluation of the robustness with the Square Attack

[max-andr]

Dear authors,

In our recent paper, we evaluated the robustness of your models using the Square Attack (a black-box attack based on random search). The adversarial accuracy we obtained came out to be far below the claimed adversarial accuracy. We formulated our findings in Section 5.2.

Here is also the robust.ml claim about our findings:
robust-ml/robust-ml.github.io#15

Our repository contains the code and steps needed to reproduce our evaluation. Please let us know if you disagree with the results that we obtained.

Best,
Maksym