Skip to content

jpwiig/simpleScraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Simple web scraper

This is a simple webscraper to be able to scrape html content from website as a bare mininum to either save it to a file, or to print it to a terminal, you can think of the first version of the script as a very simple version of Curl with some extra spice

How to use:

there are no dependcies right now (only OS and requests)

you simply:

  1. clone the repo git clone https://github.com/jpwiig/simpleScraper && cd simpleScraper
  2. run the main.py with the website you want to curl.

example: main.py https://www.nrk.no, the program will help you with the rest


be responsible and everything.

keep it safe!

Features i want to add:

  • a way to get images and videos from websites (wget like)
  • propper installation
  • more detailed error messages
  • able to show headers

If you see any crazy thing that shouldnt be there, please let me know, i will check it out! PRs and issues are open!

About

a simple web scraper by terminal

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages