Skip to content

erikgregorywebb/scheduled-jobs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 

Repository files navigation

Scheduled Jobs

Name Type Frequency Description Job
apple-app-store.py Python Daily Scrapes top 100 free Finance apps from the Apple App Store, dumps to S3. Job 1
freddie-mac-rates.py Python Daily Scrapes mortgage rates published on the home page of Freddie Mac, dumps to S3. Job 1
mormon-reddit-submissions.py Python Daily Grabs the latest 250 posts across the "mormon" subreddits, dumps to S3. Job 1
mormon-reddit-comments.py Python Every 4 Hours Grabs the latest 250 comments per "mormon" subreddit, dumps to S3. Job 2
newsapi-top-headlines.py Python Daily Collects the top headlines from the day from the News API, dumps to S3. Job 1
spotify-playlist-history.py Python Daily Scrapes track list for Spotify's Rap Cavier playlist, dumps to S3. Job 1
spotify-podcast-charts.py Python Daily Collects the Spotify podcast charts, dumps to S3. Job 1
kworb-scrape.py Python Daily Grabs kworb artist, listener, and chart objects, dumps to S3. Job 1
gila-buttes-parcels.r R Daily Scrapes details for ~280 parcels in the Gila Buttes subdivision in Casa Grande, Arizona, dumps to S3. Job 3
lds-meetinghouses-v2.r R Daily Scrapes LDS meetinghouse locations and unit assignments, dumps to S3. Job 3
zillow-scrape.r R Daily Scrapes Zillow's basic, zipcode-level ZHVF file, dumps to S3. Job 3
go-fund-me.r R Daily Scrapes GoFundMe's trending fundraiser list, dumps to S3. Job 3
s3-snapshots.py Python Daily Saves snapshot of all files and directories within S3 bucket, dumps to S3. Job 1
tsa-passenger-volumes.r R Daily Scrapes daily passenger volume counts reported by the TSA, dumps to S3. Job 3
mta-lift-status.r R Daily Scrapes elevator and escalator status logs from NY's Metro North Railroad, dumps to S3. Job 3
nyc-311-requests.py Python Daily Scrapes 10k of the latest NYC 311 requests, dumps to S3. Job 1
nyrr-races.r R Daily Scrapes a list of upcoming races hosted by NYRR, dumps to S3. Job 3
ksl-rentals.r R Daily Scrapes KSL's rental unit listings in Utah County, dumps to S3. Job 3
travel-advisories.r R Daily Scrapes the US State Department's travel advisory list, dumps to S3. Job 3
hacker-news.r R Daily Scrapes the first ten pages of YC's Hacker News, dumps to S3. Job 3

About

Collection of jobs scheduled to scrape data.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published