How to reproduce the experiment of Figure 13 in "Learning Transferable Visual Models From Natural Language Supervision"

[wendioooo]

I tried to reproduce Figure 13 which compared CLIP zero-shot transfer and ResNet101 on banana class of the natural distribution shift datasets. However, my result has a gap between the results in the paper, I got only 23% on Imagenet-a.
I think there might be something wrong with the logic of my code. I used zeroshot transfer directly on the banana data, calculated the similarity of the banana image on the text of 1000 ImageNet categories, and then calculated the accuracy. I have not found the code and detailed implementation method of this part of the experiment. How did you conduct this experiment?

[rom1504]

https://github.com/LAION-AI/CLIP_benchmark should be able to reproduce numbers