Skip to content

๐Ÿ•ท๐Ÿ•ธ Spiders to crawl ads of houses and apartments. ๐Ÿ  ๐Ÿข

License

Notifications You must be signed in to change notification settings

andreoliwa/scrapy-tegenaria

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Tegenaria

Scrapy spiders to collect and browse different items (so far, apartment ads).

How to setup the dev environment

TODO

Metrics

Stories in Ready Throughput Graph

Quickstart

First, set your app's secret key as an environment variable. For example, example add the following to .bashrc or .bash_profile.

export TEGENARIA_SECRET='something-really-secret'

Then run the following commands to bootstrap your environment.

git clone https://github.com/andreoliwa/scrapy-tegenaria
cd tegenaria
pip install -r requirements/dev.txt
flask run

You will see a pretty welcome screen.

Once you have installed your DBMS, run the following to create your app's database tables and perform the initial migration:

flask db init
flask db migrate
flask db upgrade
flask run

Deployment

In your production environment, make sure the TEGENARIA_ENV environment variable is set to "prod".

Shell

To open the interactive shell, run

flask shell

By default, you will have access to app, db, and the User model.

Running Tests

To run all tests, run

flask test

Migrations

Whenever a database migration needs to be made. Run the following commands:

flask db migrate

This will generate a new migration script. Then run:

flask db upgrade

To apply the migration.

For a full migration command reference, run flask db --help.

About

๐Ÿ•ท๐Ÿ•ธ Spiders to crawl ads of houses and apartments. ๐Ÿ  ๐Ÿข

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published