Add topdown kwarg to AbstractFileSystem.walk()

[ianthomas23]

This PR adds a new boolean kwarg topdown to AbstractFileSystem.walk(). Closes #1042.

I have extended the existing test_walk to cover both topdown=True and topdown=False.

Code produces the same output as os.walk which I have manually verified. I'll reproduce my test here in case it is of interest to anyone else.

Test directory structure:

$ cd ~/Desktop/temp && mkdir -p a/c/e a/d b && touch a/a0 b/b0 a/c/c0 a/d/d0 a/c/e/e0
$ tree 
# .
# ├── a
# │   ├── a0
# │   ├── c
# │   │   ├── c0
# │   │   └── e
# │   │       └── e0
# │   └── d
# │       └── d0
# └── b
#     └── b0

Test code:

import fsspec
import os

directory = "/Users/iant/Desktop/temp"
fs = fsspec.filesystem('file')

for topdown in [True, False]:
    for lib, name in zip([fs, os], ["fsspec", "os"]):
        print(f"topdown={topdown} {name}")
        for paths, dirs, files in lib.walk(directory, topdown=topdown):
            print(f"  paths={paths} dirs={dirs} files={files}")

Output:

$ python test.py 
topdown=True fsspec
  paths=/Users/iant/Desktop/temp dirs=['a', 'b'] files=[]
  paths=/Users/iant/Desktop/temp/a dirs=['c', 'd'] files=['a0']
  paths=/Users/iant/Desktop/temp/a/c dirs=['e'] files=['c0']
  paths=/Users/iant/Desktop/temp/a/c/e dirs=[] files=['e0']
  paths=/Users/iant/Desktop/temp/a/d dirs=[] files=['d0']
  paths=/Users/iant/Desktop/temp/b dirs=[] files=['b0']
topdown=True os
  paths=/Users/iant/Desktop/temp dirs=['a', 'b'] files=[]
  paths=/Users/iant/Desktop/temp/a dirs=['c', 'd'] files=['a0']
  paths=/Users/iant/Desktop/temp/a/c dirs=['e'] files=['c0']
  paths=/Users/iant/Desktop/temp/a/c/e dirs=[] files=['e0']
  paths=/Users/iant/Desktop/temp/a/d dirs=[] files=['d0']
  paths=/Users/iant/Desktop/temp/b dirs=[] files=['b0']
topdown=False fsspec
  paths=/Users/iant/Desktop/temp/a/c/e dirs=[] files=['e0']
  paths=/Users/iant/Desktop/temp/a/c dirs=['e'] files=['c0']
  paths=/Users/iant/Desktop/temp/a/d dirs=[] files=['d0']
  paths=/Users/iant/Desktop/temp/a dirs=['c', 'd'] files=['a0']
  paths=/Users/iant/Desktop/temp/b dirs=[] files=['b0']
  paths=/Users/iant/Desktop/temp dirs=['a', 'b'] files=[]
topdown=False os
  paths=/Users/iant/Desktop/temp/a/c/e dirs=[] files=['e0']
  paths=/Users/iant/Desktop/temp/a/c dirs=['e'] files=['c0']
  paths=/Users/iant/Desktop/temp/a/d dirs=[] files=['d0']
  paths=/Users/iant/Desktop/temp/a dirs=['c', 'd'] files=['a0']
  paths=/Users/iant/Desktop/temp/b dirs=[] files=['b0']
  paths=/Users/iant/Desktop/temp dirs=['a', 'b'] files=[]

[martindurant]

Perfect, thank you.