Skip to content

Latest commit

 

History

History
24 lines (14 loc) · 697 Bytes

README.md

File metadata and controls

24 lines (14 loc) · 697 Bytes

Crawling-bot

You can download a CSV file easily using this Crawling-bot. One of the primary goals of this project is to integrate Azure OpenAI to generate meaningful data based on the user's current needs.

How to Use

  1. Install required packages:
pip install -r requirements.txt

The requirements.txt file contains the dependencies needed for this project.

describe function

  • crawling.py: Crawls content lists and retrieves data as a CSV file.
  • detail-brute-force-crawling.py: Crawls content details and comments, then saves the data as a CSV file.
  • requirements.txt: Lists project dependencies.
  • crawling_result: Stores the resulting CSV files.