this is a web crawl project
Open folder Crawler, main script: crawler list of jobs.py file Data.json stores some info's ( "Page informations": {'Domain',Q',Number Of Pages', 'Crawled Urls',Not crawled Urls'}) file jobdata.json stores job crawled data ( "Job informations": {'url','Τίτλος Δουλειάς','Εταιρία:', 'Απασχοληση','Περιοχή','kodikos thesis'})
what next: finish store data on json - not need any longer - storing data to database (mysql) ongoing change code from BeautifulSoup to Scrapy