news-crawler + Elasticsearch

Crawling news from daum and naver main.

Getting Started

Run docker image and that's it!

How it works

This python script uses tools below:

• Selenium
• Elasticsearch
• BeautifulSoup4

By running this script, news title and provider information is gathered. And this script send JSON document to Elasticsearch in AWS.

More information

• Elasticsearch
  • index : naver-news-article-v1
  • index : daum-news-article-v1
• Kibana