Name | Type | Frequency | Description | Job |
---|---|---|---|---|
apple-app-store.py |
Python | Daily | Scrapes top 100 free Finance apps from the Apple App Store, dumps to S3. | Job 1 |
freddie-mac-rates.py |
Python | Daily | Scrapes mortgage rates published on the home page of Freddie Mac, dumps to S3. | Job 1 |
mormon-reddit-submissions.py |
Python | Daily | Grabs the latest 250 posts across the "mormon" subreddits, dumps to S3. | Job 1 |
mormon-reddit-comments.py |
Python | Every 4 Hours | Grabs the latest 250 comments per "mormon" subreddit, dumps to S3. | Job 2 |
newsapi-top-headlines.py |
Python | Daily | Collects the top headlines from the day from the News API, dumps to S3. | Job 1 |
spotify-playlist-history.py |
Python | Daily | Scrapes track list for Spotify's Rap Cavier playlist, dumps to S3. | Job 1 |
spotify-podcast-charts.py |
Python | Daily | Collects the Spotify podcast charts, dumps to S3. | Job 1 |
kworb-scrape.py |
Python | Daily | Grabs kworb artist, listener, and chart objects, dumps to S3. | Job 1 |
gila-buttes-parcels.r |
R | Daily | Scrapes details for ~280 parcels in the Gila Buttes subdivision in Casa Grande, Arizona, dumps to S3. | Job 3 |
lds-meetinghouses-v2.r |
R | Daily | Scrapes LDS meetinghouse locations and unit assignments, dumps to S3. | Job 3 |
zillow-scrape.r |
R | Daily | Scrapes Zillow's basic, zipcode-level ZHVF file, dumps to S3. | Job 3 |
go-fund-me.r |
R | Daily | Scrapes GoFundMe's trending fundraiser list, dumps to S3. | Job 3 |
s3-snapshots.py |
Python | Daily | Saves snapshot of all files and directories within S3 bucket, dumps to S3. | Job 1 |
tsa-passenger-volumes.r |
R | Daily | Scrapes daily passenger volume counts reported by the TSA, dumps to S3. | Job 3 |
mta-lift-status.r |
R | Daily | Scrapes elevator and escalator status logs from NY's Metro North Railroad, dumps to S3. | Job 3 |
nyc-311-requests.py |
Python | Daily | Scrapes 10k of the latest NYC 311 requests, dumps to S3. | Job 1 |
nyrr-races.r |
R | Daily | Scrapes a list of upcoming races hosted by NYRR, dumps to S3. | Job 3 |
ksl-rentals.r |
R | Daily | Scrapes KSL's rental unit listings in Utah County, dumps to S3. | Job 3 |
travel-advisories.r |
R | Daily | Scrapes the US State Department's travel advisory list, dumps to S3. | Job 3 |
hacker-news.r |
R | Daily | Scrapes the first ten pages of YC's Hacker News, dumps to S3. | Job 3 |
-
Notifications
You must be signed in to change notification settings - Fork 1
erikgregorywebb/scheduled-jobs
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
About
Collection of jobs scheduled to scrape data.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published