Yahoo Auction JP Product Information Crawler This project includes following tools:
- Crawler written with scrapy
- csv_fix.py to remove abundant blank lines when use in Windows
- csv_cplit.py to divide the result CSV file into 300-line CSV files
Install python 3
Run these commands in terminal or cmd:
scrapy crawl products -a author_url="https://auctions.yahoo.co.jp/seller/verifymarche"
py csv_fix.py (in windows only)
py csv_split.py
py create_zips.py
MIT License