spotify-data

A repository for tracking Spotify artist changes over time.

How it works

Every day the Sync workflow will run a Ruby script that hits the Spotify API with the artist ids in the artists.json file. The data is saved by id in the output directory, committed and pushed to the repository.

Adding artists

Artist ids can be found by copying & pasting a link to the artist's profile from the Spotify web or desktop applications, which will be in the format: https://open.spotify.com/artist/:artistId. This id can then be added to the array in the artists.json file.

Development

To run the sync script locally, you'll need Ruby installed. You'll also need a Spotify Web API key.

bundle install
CLIENT_IDS=abc123 CLIENT_SECRETS=xyz456 ruby scripts/sync.rb

Expanding the git history

The git history for an artist can be expanded into a JSON file that contains an object for each snapshot of the artist's metadata. This file can be uploaded to a database or dropped into a charting library to visualize the changes over time.

To expand an artist's history, run the expand_artist_history.rb script with the id of the artist as a command-line argument:

ruby scripts/expand_artist_history.rb 6lcwlkAjBPSKnFBZjjZFJs

Building a SQLite database

The git history for all artists can be iterated to build a SQLite database for embedding into your application, or using for local development with the arthistory app.

To build the SQLite database, run the build_sqlite_database.rb script. You can optionally provide skip and take command-line arguments to generate a smaller subset of data for testing.

# Generates a database for the entire git history
ruby scripts/build_sqlite_database.rb

# Only generate snapshots for the last 20 commits
ruby scripts/build_sqlite_database.rb --take 20

# Run with logging output
ruby scripts/build_sqlite_database.rb --verbose

Reference

This repository was inspired by a project for tracking Spotify changes that I didn't end up finishing. A few years later, Git scraping: track changes over time by scraping to a Git repository by Simon Willison showed up on HackerNews and inspired me to implement the core of my idea.