Skip to content

Using scrapy lib to scrape and then save data to postgress then visualize in htlm server all dockerized.

Notifications You must be signed in to change notification settings

stehlfi1/Scrapy-reality-project

Repository files navigation

Scrapy-reality-project

Utilizing Scrapy to scrape data, which is then stored in a PostgreSQL database and visualized through an HTML-based web server, all within a Dockerized environment.

How to Launch:

Prerequisites

  1. Install Docker on your system.
  2. Install Git (if not already installed).

Steps:

  1. Clone the repository:

    git clone https://github.com/stehlfi1/Scrapy-reality-project.git
  2. Navigate to the project directory:

    cd Scrapy-reality-project
  3. Build and Launch the Docker Compose:

    docker-compose up --build
  4. Access App:

    • Open your web browser and go to http://localhost:8070.
  5. To Stop the Services:

    docker-compose down

Troubleshooting:

  1. If facing authentication errors, make sure to restart the Docker containers.
    docker-compose down
    docker-compose up

About

Using scrapy lib to scrape and then save data to postgress then visualize in htlm server all dockerized.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published