Azure macOS job is failing + "pip search" throttling

[uranusjr]

I recall @pradyunsg made the observation under a PR, but I can’t locate it now. (Edit: Found it #9017 (comment))

The specific failure is

E       AssertionError: temp files not expected
-- created: -------------------
  tmp/com.apple.dyld
          sh-B4B247918EFC62F0DED3F9B92246A6C10C7FECE20649ECEA30D36D86DEF79029.closure  (39976 bytes)
          uname-8487F8338686B8B010BF7BAEFA1A50C27FCA318D2FED3E6B363D8D561583F290.closure  (36168 bytes)

(The list of files change depending on the test case.)

I did some research, it seems like com.apple.dyld is a storage called shared dyld cache and is a cache mechanism used by Apple’s dynamic linker (dyld3). All information I’ve found so far are about iOS, but it shouldn’t be surprising if Apple port that to macOS. This post from 2018 talks about the mechanism in more detail.

I’m not sure why the Azure images use this mechanism (my machine running Catelina doesn’t seem to have them), but it seems like we should be safe to just ignore the whole com.apple.dyld directory when we check for temporary files?

[pradyunsg]

That, and it looks like our CI pipelines are hitting the newly introduced rate limits for PyPI's XMRPC API. :(

Context: pypi/warehouse#4321 (comment)

[pradyunsg]

#9051 fixes the search issues. And, also surfaces that the underlying issue with the MacOS build affects all the tests, not just the build-env ones. :(

[pradyunsg]

I spent a fair bit of time looking into this, and I can't really find out what the heck is going on and why we're seeing those files.

I'm experimenting with #9053, but opinions and thoughts on how to workaround this w/o access to scripttest are welcome. :)

[pradyunsg]

@pfmoore pypa/scripttest#25