Skip to content

Romissevd/WebCrawler_FR

Repository files navigation

#Scripts for update data

##Wiki France parser command

graber/france/wiki.py [-f] [-s /path/to/insee/file.csv] [-r 'insee+12345'] [-l 'https://fr.wikipedia.org/wiki/Paris']

-f force update
-s source of insee data
-r request to wiki with search
-l link to wiki page

#Environment

install docker with docker-compose

##Run dev environment

cd /path/to/project/root
docker-compose -f ./docker/docker-compose-dev.yml up

Clear logs

sudo rm -rf ./log/*.log

##Update project in prod

cd /project/gwparser/batch
git pull
/project/gwparser/batch/update

#Run tests in docker

docker exec -it docker_wiki_parser_python_1 python /app/tests/wiki_fr_parser.py

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published