! Create a virtualenv if making changes to this project.
Build and install guide.
venv/bin/python setup.py build
venv/bin/python setup.py installthen:
python parser.py- bbcgoodfood.com
Simple usage:
from recipe_scrapers import scrape_me
# give the url as a string, it can be url from any site listed below
scraper = scrape_me('https://www.allrecipes.com/recipe/158968/spinach-and-feta-turkey-burgers/')
# Q: What if the recipe site I want to extract information from is not listed below?
# A: You can give it a try with the wild_mode option! If there is Schema/Recipe available it will work just fine.
scraper = scrape_me('https://www.feastingathome.com/tomato-risotto/', wild_mode=True)
scraper.title()
scraper.total_time()
scraper.yields()
scraper.ingredients()
scraper.instructions()
scraper.image()
scraper.host()
scraper.links()Note: scraper.links() returns a list of dictionaries containing all of the <a> tag attributes. The attribute names are the dictionary keys.
- https://www.750g.com
- https://www.acouplecooks.com
- https://allrecipes.com/
- https://ambitiouskitchen.com/
- https://archanaskitchen.com/
- https://averiecooks.com/
- https://bbc.com/
- https://bbc.co.uk/
- https://bbcgoodfood.com/
- https://bettycrocker.com/
- https://bonappetit.com/
- https://bowlofdelicious.com/
- https://budgetbytes.com/
- https://chefkoch.de/
- https://closetcooking.com/
- https://cookieandkate.com/
- https://cookpad.com/
- https://cookstr.com/
- https://copykat.com/
- https://countryliving.com/
- https://cuisineaz.com/
- https://cybercook.com.br/
- https://delish.com/
- https://domesticate-me.com/
- https://eatsmarter.com/
- https://eatsmarter.de/
- https://eatwhattonight.com/
- https://epicurious.com/
- https://recipes.farmhousedelivery.com/
- https://fifteenspatulas.com/
- https://finedininglovers.com/
- https://fitmencook.com/
- https://food.com/
- https://foodnetwork.com/
- https://foodrepublic.com/
- https://geniuskitchen.com/
- https://giallozafferano.it/
- https://gimmesomeoven.com/
- https://gonnawantseconds.com/
- https://gousto.co.uk/
- https://greatbritishchefs.com/
- https://halfbakedharvest.com/
- https://www.hassanchef.com/
- https://www.heb.com/
- https://heinzbrasil.com.br/
- https://hellofresh.com/
- https://hellofresh.co.uk/
- https://www.hellofresh.de/
- https://hostthetoast.com/
- https://101cookbooks.com/
- https://receitas.ig.com.br/
- https://inspiralized.com/
- https://jamieoliver.com/
- https://justbento.com/
- https://kennymcgovern.com/
- https://www.kingarthurbaking.com
- https://kochbar.de/
- https://kuchnia-domowa.pl/
- http://livelytable.com/
- https://lovingitvegan.com/
- https://lecremedelacrumb.com/
- https://marmiton.org/
- https://matprat.no/
- http://mindmegette.hu/
- https://minimalistbaker.com/
- https://misya.info/
- https://momswithcrockpots.com/
- http://motherthyme.com/
- https://mybakingaddiction.com/
- https://myrecipes.com/
- https://healthyeating.nhlbi.nih.gov/
- https://nourishedbynutrition.com/
- https://nutritionbynathalie.com/blog
- https://cooking.nytimes.com/
- https://ohsheglows.com/
- https://www.panelinha.com.br/
- https://paninihappy.com/
- https://przepisy.pl/
- https://purelypope.com/
- https://realsimple.com/
- https://recipietineats.com/
- https://seriouseats.com/
- https://simplyquinoa.com/
- https://simplyrecipes.com/
- https://simplywhisked.com/
- https://skinnytaste.com/
- https://southernliving.com/
- https://spendwithpennies.com/
- https://steamykitchen.com/
- https://streetkitchen.hu/
- https://sunbasket.com/
- https://sweetpeasandsaffron.com/ <https://sweetpeasandsaffron.com/>
- https://tastesoflizzyt.com
- https://tasteofhome.com
- https://tasty.co
- https://tastykitchen.com/
- https://thehappyfoodie.co.uk/
- https://thekitchn.com/
- https://thenutritiouskitchen.co/
- https://thepioneerwoman.com/
- https://thespruceeats.com/
- https://thevintagemixer.com/
- https://thewoksoflife.com/
- https://tine.no/
- https://tudogostoso.com.br/
- https://twopeasandtheirpod.com/
- https://vegolosi.it/
- https://watchwhatueat.com/
- https://whatsgabycooking.com/
- https://en.wikibooks.org/
- https://yummly.com/
Part of the reason I want this open sourced is because if a site makes a design change, the scraper for it should be modified.
If you spot a design change (or something else) that makes the scraper unable to work for a given site - please fire an issue asap.
If you are programmer PRs with fixes are warmly welcomed and acknowledged with a virtual beer.
- Open an Issue providing us the site name, as well as a recipe link from it.
- You are a developer and want to code the scraper on your own:
- If Schema is available on the site - you can do this
- Otherwise, scrape the HTML - like this
Assuming you have python3 installed, navigate to the directory where you want this project to live in and drop these lines
git clone git@github.com:hhursev/recipe-scrapers.git &&
cd recipe-scrapers &&
python3 -m venv .venv &&
source .venv/bin/activate &&
pip install -r requirements-dev.txt &&
pre-commit install &&
python -m coverage run -m unittest &&
python -m coverage reportIn case you want to run a single unittest for a newly developed scraper
python -m coverage run -m unittest tests.test_myscraper- How do I know if a website has a Recipe Schema? Run in python shell:
from recipe_scrapers import scrape_me
scraper = scrape_me('<url of a recipe from the site>', wild_mode=True)
# if no error is raised - there's schema available:
scraper.title()
scraper.instructions() # etc.All the contributors that helped improving the package. You are awesome!