Repositories list

• web-search

  Public
  Code that drives the public web-based tools for the Media Cloud Online News Archive and Directory.

  social-mediaresearch-toolsocial-science+ 1media-analytics

  JavaScript

  •

  Apache License 2.0

  •16•10•64•0•Updated Mar 10, 2025Mar 10, 2025

• mc-providers

  Public
  Internal library to allow querying multiple media platforms with a consistent API.

  Python

  •4•1•9•0•Updated Mar 6, 2025Mar 6, 2025

• story-indexer

  Public
  The core pipeline used to ingest online news stories in the Media Cloud archive.

  Python

  •

  Apache License 2.0

  •6•4•44•6•Updated Mar 2, 2025Mar 2, 2025

• ansible-elasticsearch

  Public
  Ansible playbook for Elasticsearch

  Ruby

  •

  Other

  •859•0•0•0•Updated Feb 22, 2025Feb 22, 2025

• system-dev-ops

  Public
  devops tools

  Python

  •

  Apache License 2.0

  •1•0•1•0•Updated Feb 21, 2025Feb 21, 2025

• sc-buffet

  Public
  Sous-chef buffet - Self-service data access for sous-chef.

  Python

  •1•0•5•1•Updated Jan 31, 2025Jan 31, 2025

• directory-issues

  Public
  UNDER CONSTRUCTION - A package containing a library of issue validators in a flexibly deployable wrapper.

  Jupyter Notebook

  •2•0•9•0•Updated Jan 31, 2025Jan 31, 2025

• es-tools

  Public
  Elasticsearch tools developed by the Media Cloud project

  Python

  •

  Apache License 2.0

  •1•0•0•0•Updated Jan 25, 2025Jan 25, 2025

• metadata-lib

  Public
  How Media Cloud approaches extracting metadata from online news stories

  Python

  •

  Apache License 2.0

  •5•12•6•0•Updated Dec 22, 2024Dec 22, 2024

• api-client

  Public
  Public client for consuming content from the Media Cloud Online News Archive & Directory.

  Python

  •

  Apache License 2.0

  •29•72•4•1•Updated Dec 10, 2024Dec 10, 2024

• rss-fetcher

  Public
  Intelligently fetch lists of URLs from a large collection of RSS Feeds as part of the Media Cloud Directory.

  Python

  •

  Apache License 2.0

  •6•6•12•1•Updated Dec 5, 2024Dec 5, 2024

• news-search-api

  Public
  Internal API server that offers search access to the Media Cloud Online News Archive (in Elasticsearch).

  Python

  •

  GNU Affero General Public License v3.0

  •4•1•10•0•Updated Oct 25, 2024Oct 25, 2024

• sous-chef

  Public
  Configurable Data Analytics Pipeline

  sous-chef

  Python

  •0•1•9•0•Updated Oct 21, 2024Oct 21, 2024

• mediacloud-news-client

  Public archive
  An internal client library to access the new Mediacloud news archive search.

  Python

  •

  Apache License 2.0

  •2•0•3•1•Updated Oct 10, 2024Oct 10, 2024

• feed_seeker

  Public
  Find rss, atom, xml, and rdf feeds on webpages

  Python

  •

  MIT License

  •13•30•4•1•Updated Oct 10, 2024Oct 10, 2024

• sitemap-tools

  Public
  simple toolkit of tools for consuming sitemaps

  Python

  •

  Apache License 2.0

  •1•4•2•0•Updated Oct 9, 2024Oct 9, 2024

• mc-manage

  Public
  Python

  •0•0•0•0•Updated Oct 8, 2024Oct 8, 2024

• system-metrics

  Public
  Daily performance metrics for the mediacloud application

  sous-chef

  Python

  •0•0•1•0•Updated Sep 20, 2024Sep 20, 2024

• cliff-api-client

  Public
  A Python client for the CLIFF geoparsing tool

  Python

  •

  MIT License

  •5•5•0•1•Updated May 21, 2024May 21, 2024

• wayback-news-client

  Public
  A client library to access the Wayback Machine news archive search.

  Python

  •

  Apache License 2.0

  •2•4•1•0•Updated Dec 15, 2023Dec 15, 2023

• web-tools

  Public archive
  The shared repository for Media Cloud web apps (Explorer, Source Manager, Topic Mapper)

  flaskreactjscivic-tech+ 3media-analyticspythonmedia-cloud

  JavaScript

  •

  Apache License 2.0

  •30•64•31•4•Updated Dec 14, 2023Dec 14, 2023

• api-tutorial-notebooks

  Public
  A set of jupyter notebooks demonstrating how to use the Media Cloud API.

  Jupyter Notebook

  •14•37•0•0•Updated Dec 13, 2023Dec 13, 2023

• backend

  Public archive
  Media Cloud is an open source, open data platform that allows researchers to answer quantitative questions about the content of online media.

  medianewspapercontent-analysis+ 3news-medianewsmediaonline-news

  Python

  •

  GNU Affero General Public License v3.0

  •88•283•131•25•Updated Nov 20, 2023Nov 20, 2023

• a-catchall-page

  Public
  Dokku app that serves a static HTML catch-all page, displayed for bad domains

  HTML

  •0•0•0•0•Updated Oct 25, 2023Oct 25, 2023

• cliff-homepage

  Public
  A simple homepage for the CLIFF project

  HTML

  •

  MIT License

  •1•1•0•0•Updated May 30, 2023May 30, 2023

• nyt-news-labeler

  Public
  Tag news stories based on models trained on the NYT corpus.

  Python

  •

  Apache License 2.0

  •13•42•1•6•Updated Mar 1, 2023Mar 1, 2023

• sentence-splitter

  Public
  Text to sentence splitter using heuristic algorithm by Philipp Koehn and Josh Schroeder.

  nlptokenizersentence-splitting

  Python

  •

  Other

  •30•241•2•2•Updated Nov 7, 2022Nov 7, 2022

• glimpse

  Public archive
  Get a glimpse of attention to a topic on social media.

  Python

  •

  Apache License 2.0

  •2•2•8•0•Updated Sep 19, 2022Sep 19, 2022

• word-embeddings-server

  Public archive
  Helpful micro-service to return results from word2vec models

  Python

  •

  MIT License

  •4•2•0•0•Updated Jul 29, 2022Jul 29, 2022

• cliff-annotator

  Public archive
  A lightweight server to allow HTTP requests to the Stanford Named Entity Recognized and a heavily modified CLAVIN geoparser.

  Java

  •

  Apache License 2.0

  •36•119•9•10•Updated May 20, 2022May 20, 2022