README.md

PubMed

Introduction:

Web scraping script was created to extract articles information from PubMed database https://www.ncbi.nlm.nih.gov/pubmed/.

Data is stored in MongoDB first then extracted to conduct data preprcoessing, manipulation and visualizaiton. More information could be found on http://woodenleaves.com/pages/pubmed.html

1. PubMed_Scraping.py:

Tools:

Python(Selenium, BeautifulSoup, Requests, Multiprocessing, Pandas, pymongo, re, bokeh, matplotlib)

MongoDB

ECharts.js

2. PubMed.ipynb:

Data preprocessing, statistical analysis and data visualizaton

3. Demo