Remove a stray file (#5242)

While re-reviewing #5083 to find the source of a performance regression
across the board on that commit I noticed that an empty file test.py was
accidently committed to the root of the repo. This commit fixes that
oversight and removes the file.