From 9995ebc2413af7365b82d1b2daabe646fcb483ea Mon Sep 17 00:00:00 2001 From: Will Moss Date: Sun, 25 Aug 2024 01:17:51 +0700 Subject: [PATCH] First commit --- .gitignore | 2 + LICENSE | 21 ++++ README.md | 202 ++++++++++++++++++++++++++++++++++++++ dawson | 179 +++++++++++++++++++++++++++++++++ dawson.py | 177 +++++++++++++++++++++++++++++++++ requirements.txt | 2 + scripts/local-install.sh | 32 ++++++ scripts/remote-install.sh | 37 +++++++ 8 files changed, 652 insertions(+) create mode 100644 .gitignore create mode 100644 LICENSE create mode 100644 README.md create mode 100755 dawson create mode 100644 dawson.py create mode 100644 requirements.txt create mode 100755 scripts/local-install.sh create mode 100755 scripts/remote-install.sh diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..03f0c79 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +scripts/_* +.aliases diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..07d7f86 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2024 Will Moss + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..c89c7f3 --- /dev/null +++ b/README.md @@ -0,0 +1,202 @@ +

+

Dawson

+

+ Track your project's statistics on Hacker News and Github, and get notified on every new interaction +

+

+ Introduction - + Features - + Install +

+

+ +## Table of Contents + +- [Introduction](#introduction) +- [Features](#features) +- [Install](#install) +- [Examples](#examples) +- [Troubleshoot](#troubleshoot) +- [Credits](#credits) + +## Introduction + +Dawson is a tiny utility that enables you to track your project's submission on Hacker News, and your project's repository on Github. +On every new comment, upvote, fork, issue, or star, Dawson will send you a desktop notification so you can quickly react and anticipate. + +I decided to create that tool after noticing that my latest project ([Isaiah](https://github.com/will-moss/isaiah)) had made it to the +front page of Hacker News, and brought hundreds of reactions and comments. I wanted to be able to track everything without having to refresh +my browser every other minute, and quickly reply to questions and suggestions. That's why I built Dawson + +## Features + +Dawson has these features implemented : +- Track any given Hacker News post or Github repository +- Send a desktop notification on every new comment, upvote, fork, issue, or star +- Open the monitored page when clicking on notifications +- Customize the monitoring (frequency, title, sound, number of notifications, etc.) + +Everything fits into one small Python file, and you can either install Dawson, or copy the script and run it locally. + +## Install + +Before proceeding, please make sure that your system meets the following requirements: +- Python 3.x is installed, and available as the `python3` executable +- Pip 3.x is installed, and available as the `pip3` executable +- On Mac OS, read the troubleshooting section + +Then, you may choose one of the options below. + +#### Using the remote install script + +A remote install script was created to help you install Dawson in one line, from your terminal: + +> As always, check the content of every file you pipe in bash + +```bash +# Run the install script +curl https://raw.githubusercontent.com/will-moss/dawson/master/scripts/remote-install.sh | bash + +# Dawson should be available now +dawson -h +``` + +#### Using the local install script + +A local install script was created to help you install Dawson in a few lines, from your terminal: + +```bash +# Retrieve the code +git clone https://github.com/will-moss/dawson +cd dawson + +# Run the local install script +./scripts/local-install.sh + +# Dawson should be available now +dawson -h +``` + +#### Manual install + +Dawson being nothing more than a Python script, you can install it manually with the following commands: + +```bash +# Retrieve the code +git clone https://github.com/will-moss/dawson +cd dawson + +# Install the two dependencies +pip3 install -r requirements.txt + +# Option 1 : Run Dawson using the Python executable +python3 dawson.py -h + +# Option 2 : Run Dawson using the executable (a Python script with a shebang) +./dawson -h +``` + +## Examples + +Please find below a few examples to help you get started with Dawson. + +#### Track your project's submission on Hacker News + +```bash +# Monitor for changes every 10 seconds +dawson -u https://news.ycombinator.com/item?id= -f 10 + +# Using Python +python3 dawson.py -u https://news.ycombinator.com/item?id= -f 10 +``` + +#### Track your project's repository on Github + +```bash +# Monitor for changes every 30 seconds +dawson -u https://github.com// -f 30 -T + +# Using Python +python3 dawson -u https://github.com// -f 30 -T +``` + +#### Track your project on both Hacker News and Github, with custom settings, and background jobs + +```bash +# Every process runs in the background, emits no notification sound, and has its own notification title +dawson -u https://news.ycombinator.com/item?id= -f 10 -s -t "Hacker News - Activity recorded" & +dawson -u https://github.com// -f 10 -T -s -t "Github - Activity recorded" & +``` + +#### Run Dawson in the background using screen + +```bash +# Create a screen dedicated to Dawson +screen -S dawson + +# Run Dawson +dawson -u https://news.ycombinator.com/item?id= -f 10 + +# Leave the screen instance in the background + +``` + +## Troubleshoot + +Should you encounter any issue running Dawson, please refer to the following common questions, and problems with their solutions. + +#### Can I use Dawson to monitor multiple projects at the same time? + +Dawson can monitor one project at a time, but you can run multiple instances of Dawson in parallel. + +For example, you may want to track your project on both Hacker News and Github. +To do so, you should run Dawson two times : once for Hacker News, and once for Github. +And you can, definitely, run Dawson in parallel as many times as wanted to track multiple projects at the same time. + +I would recommend using `screen` or native background jobs for that purpose, and make sure that you don't hit API rate limits. + +#### Dawson fails at monitoring my project on Github + +First, please ensure that your Github repository is publicly accessible. If it isn't, +you must supply the `-T ` command-line argument. + +Second, you may have hit Github API rate limits. In this case, you should generate a Github API Token, and supply it to Dawson +using the `-T ` command-line argument. By default, the public unauthenticated Github API is restricted to 60 requests per hour, which is very low. + +Ultimately, please make sure that : +- Your monitoring frequency (`-f ` argument) isn't too small. +- Your Github's repository URL looks like so : `https://github.com//` + +#### Notifications don't work on Mac OS + +Dawson uses the great `Desktop Notifier` library to manage notifications on every operating system. + +Should you encounter an issue on Mac OS, I suggest reading : [https://desktop-notifier.readthedocs.io/en/latest/#notes-on-macos](https://desktop-notifier.readthedocs.io/en/latest/#notes-on-macos) + +For a simple explanation : +- To display notifications in the Notification Center of Mac OS, an app needs to be signed and properly configured. +- If your version of Python comes from `brew` or another package manager, it certainly isn't signed, or not configured as Mac OS expects it to be. +- The solution is to download Python from the official [python.org](https://python.org) website, run their installer, and use that version of Python to run Dawson. + +Also, because Dawson is not a packaged and signed app for Mac OS, it can't send notifications on its own. It requires Python (which is signed and packaged!). +Packaging and signing Dawson could be doable, but it would require jumping hoops and paying for an Apple Developer License. + +#### Something else + +Please feel free to open an issue, explaining what happens, and describing your environment. + +## Contribute + +If you can help in any way, please do. Here's a few ideas : +- Open an issue if you encounter a bug +- Suggest new features and improvements +- Anything else that comes to your mind! + +## Credits + +Hey hey ! It's always a good idea to say thank you and mention the people and projects that help us move forward. + +Big thanks to the creator of [Desktop Notifier](https://github.com/samschott/desktop-notifier) +for the powerful and convenient cross-os notification library. + +And don't forget to mention Dawson if it makes your life easier! diff --git a/dawson b/dawson new file mode 100755 index 0000000..54f39cd --- /dev/null +++ b/dawson @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 + +# Dependencies : +# - argparse : CLI arguments +# - requests : Fetch stats from Hacker News & Github +# - asyncio : Execution loop (desktop_notifier requirement) +# - time : Thread sleep +# - copy : Deep copy +# - desktop_notifier : Cross-OS notifications +# - datetime : Timestamped logs +# - platform : Detect whether we are on Mac OS +# - webbrowser : Open a URL in the web browser +# - signal : Event loop management (graceful termination) +# - rubicon : Event loop management (click on notification) on Mac OS +import argparse +import requests +import asyncio +import time +import copy +from desktop_notifier import DesktopNotifier, Urgency, Button, ReplyField, DEFAULT_SOUND +from datetime import datetime +import platform +import webbrowser +import signal + +# Mac OS +try: + from rubicon.objc.eventloop import EventLoopPolicy +except: + pass + +# Prints an error and a recommendation when the supplied URL argument is in the wrong format +def print_error_url(): + print("Please copy-paste a working link to the resource you'd like to monitor") + print("It should look like, either : ") + print("- https://news.ycombinator.com/item?id=") + print("- https://github.com//") + +# Prints an error when we can't retrieve stats from the Github API +def print_error_github(): + print("We weren't able to retrieve the statistics of your Github repository") + print("Please make sure that the repository is publicly accessible") + print("Also, you may consider using the argument -T to increase the number of requests we can send") + +# Prints a log line with a timestamp +def print_log(log, quiet): + if quiet: + return + print(f"{datetime.today().strftime('%Y-%m-%d %H:%M:%S')} >> {log}") + +# Entry point +async def main(): + # Args management + parser = argparse.ArgumentParser( + prog='dawson', + description="Dawson is a tiny utility that enables you to track your project's submission on Hacker News, and your project's repository on Github.\nOn every new comment, upvote, fork, issue, or star, Dawson will send you a desktop notification so you can quickly react and anticipate.", + epilog='License: MIT\nAuthor: Will Moss', + formatter_class=argparse.RawDescriptionHelpFormatter + ) + parser.add_argument('-u', '--url', required=True) + parser.add_argument('-m', '--max', type=int, required=False, default=1, help="The maximum number of notifications shown in the notification center", metavar='') + parser.add_argument('-f', '--frequency', type=float, required=False, default=10, help="The frequency, in seconds, used to check for new statistics", metavar='') + parser.add_argument('-s', '--silent', action='store_true', required=False, default=False, help="Disable the sound of notifications") + parser.add_argument('-q', '--quiet', action='store_true', required=False, default=False, help="Disable the output of log lines in the console") + parser.add_argument('-t', '--title', required=False, default="Dawson - Activity recorded", help="The title shown in the notifications", metavar='') + parser.add_argument('-T', '--token', required=False, help="The Github token used to circumvent API rate limits", metavar='') + + + # Args parsing - With graceful shutdown on Mac OS + if platform.system() == 'Darwin': + try: + args = parser.parse_args() + except: + return + else: + args = parser.parse_args() + + # Notifications management + notifier = DesktopNotifier( + app_name='Dawson', + notification_limit=args.max, + ) + + # Async management - Graceful termination + async_stop_event = asyncio.Event() + async_loop = asyncio.get_running_loop() + + async_loop.add_signal_handler(signal.SIGINT, async_stop_event.set) + async_loop.add_signal_handler(signal.SIGTERM, async_stop_event.set) + + # Checks before startup + + # - Domain + if 'news.ycombinator.com' not in args.url and 'github.com' not in args.url: + print_error_url() + exit(1) + + # - Hacker News format + if 'news.ycombinator.com' in args.url: + if len(args.url.split('?id=')) != 2: + print_error_url() + exit(1) + + # - Github format + elif 'github.com' in args.url: + if len(list(filter(None, args.url.split('github.com/')[1].split('/')))) < 2: + print_error_url() + exit(1) + + # - Frequency + + # Actual monitoring loop + monitoring_type = 'hackernews' if 'news.ycombinator.com' in args.url else 'github' + print_log('Monitoring : Start', args.quiet) + + previous_metrics = {} + while not async_stop_event.is_set(): + should_notify = False + + # Monitoring : Hacker News + if monitoring_type == 'hackernews': + resource_id = args.url.split('?id=')[1] + resource_url = f"https://hacker-news.firebaseio.com/v0/item/{resource_id}.json" + resource_data = requests.get(resource_url).json() + + new_metrics = { 'score': resource_data['score'], 'comments': resource_data['descendants'] } + + # Monitoring : Github + elif monitoring_type == 'github': + resource_id = '/'.join(args.url.split('github.com/')[1].split('/')[0:2]) + resource_url = f"https://api.github.com/repos/{resource_id}" + + headers = {} + if args.token: + headers['Authorization'] = f'Bearer {args.token}' + + resource_data = requests.get(resource_url, headers=headers).json() + + if 'id' not in resource_data: + print_error_github() + exit(1) + + new_metrics = { 'stars': resource_data['stargazers_count'], 'forks': resource_data['forks_count'], 'issues': resource_data['open_issues_count'] } + + # Guard : Initialize metrics on first run + if len(previous_metrics) == 0: + previous_metrics = copy.deepcopy(new_metrics) + continue + + # Determine whether a notification should be sent + for k in new_metrics: + if previous_metrics[k] != new_metrics[k]: + should_notify = True + break + + # Notification + if should_notify: + print_log('Monitoring : New activity', args.quiet) + previous_metrics = copy.deepcopy(new_metrics) + + lines = [ f"{k.capitalize()}: {v}" for k, v in previous_metrics.items() ] + message = "\n".join(lines) + + await notifier.send( + title=args.title, + message=message, + sound=DEFAULT_SOUND if not args.silent else None, + on_clicked=lambda: webbrowser.open(args.url, new=0, autoraise=True) + ) + + await asyncio.sleep(args.frequency) + + print_log('Monitoring : End', args.quiet) + +# Specific setup to handle click on notifications on Mac OS +if platform.system() == 'Darwin': + asyncio.set_event_loop_policy(EventLoopPolicy()) + +asyncio.run(main()) diff --git a/dawson.py b/dawson.py new file mode 100644 index 0000000..2f79a39 --- /dev/null +++ b/dawson.py @@ -0,0 +1,177 @@ +# Dependencies : +# - argparse : CLI arguments +# - requests : Fetch stats from Hacker News & Github +# - asyncio : Execution loop (desktop_notifier requirement) +# - time : Thread sleep +# - copy : Deep copy +# - desktop_notifier : Cross-OS notifications +# - datetime : Timestamped logs +# - platform : Detect whether we are on Mac OS +# - webbrowser : Open a URL in the web browser +# - signal : Event loop management (graceful termination) +# - rubicon : Event loop management (click on notification) on Mac OS +import argparse +import requests +import asyncio +import time +import copy +from desktop_notifier import DesktopNotifier, Urgency, Button, ReplyField, DEFAULT_SOUND +from datetime import datetime +import platform +import webbrowser +import signal + +# Mac OS +try: + from rubicon.objc.eventloop import EventLoopPolicy +except: + pass + +# Prints an error and a recommendation when the supplied URL argument is in the wrong format +def print_error_url(): + print("Please copy-paste a working link to the resource you'd like to monitor") + print("It should look like, either : ") + print("- https://news.ycombinator.com/item?id=") + print("- https://github.com//") + +# Prints an error when we can't retrieve stats from the Github API +def print_error_github(): + print("We weren't able to retrieve the statistics of your Github repository") + print("Please make sure that the repository is publicly accessible") + print("Also, you may consider using the argument -T to increase the number of requests we can send") + +# Prints a log line with a timestamp +def print_log(log, quiet): + if quiet: + return + print(f"{datetime.today().strftime('%Y-%m-%d %H:%M:%S')} >> {log}") + +# Entry point +async def main(): + # Args management + parser = argparse.ArgumentParser( + prog='dawson', + description="Dawson is a tiny utility that enables you to track your project's submission on Hacker News, and your project's repository on Github.\nOn every new comment, upvote, fork, issue, or star, Dawson will send you a desktop notification so you can quickly react and anticipate.", + epilog='License: MIT\nAuthor: Will Moss', + formatter_class=argparse.RawDescriptionHelpFormatter + ) + parser.add_argument('-u', '--url', required=True) + parser.add_argument('-m', '--max', type=int, required=False, default=1, help="The maximum number of notifications shown in the notification center", metavar='') + parser.add_argument('-f', '--frequency', type=float, required=False, default=10, help="The frequency, in seconds, used to check for new statistics", metavar='') + parser.add_argument('-s', '--silent', action='store_true', required=False, default=False, help="Disable the sound of notifications") + parser.add_argument('-q', '--quiet', action='store_true', required=False, default=False, help="Disable the output of log lines in the console") + parser.add_argument('-t', '--title', required=False, default="Dawson - Activity recorded", help="The title shown in the notifications", metavar='') + parser.add_argument('-T', '--token', required=False, help="The Github token used to circumvent API rate limits", metavar='') + + + # Args parsing - With graceful shutdown on Mac OS + if platform.system() == 'Darwin': + try: + args = parser.parse_args() + except: + return + else: + args = parser.parse_args() + + # Notifications management + notifier = DesktopNotifier( + app_name='Dawson', + notification_limit=args.max, + ) + + # Async management - Graceful termination + async_stop_event = asyncio.Event() + async_loop = asyncio.get_running_loop() + + async_loop.add_signal_handler(signal.SIGINT, async_stop_event.set) + async_loop.add_signal_handler(signal.SIGTERM, async_stop_event.set) + + # Checks before startup + + # - Domain + if 'news.ycombinator.com' not in args.url and 'github.com' not in args.url: + print_error_url() + exit(1) + + # - Hacker News format + if 'news.ycombinator.com' in args.url: + if len(args.url.split('?id=')) != 2: + print_error_url() + exit(1) + + # - Github format + elif 'github.com' in args.url: + if len(list(filter(None, args.url.split('github.com/')[1].split('/')))) < 2: + print_error_url() + exit(1) + + # - Frequency + + # Actual monitoring loop + monitoring_type = 'hackernews' if 'news.ycombinator.com' in args.url else 'github' + print_log('Monitoring : Start', args.quiet) + + previous_metrics = {} + while not async_stop_event.is_set(): + should_notify = False + + # Monitoring : Hacker News + if monitoring_type == 'hackernews': + resource_id = args.url.split('?id=')[1] + resource_url = f"https://hacker-news.firebaseio.com/v0/item/{resource_id}.json" + resource_data = requests.get(resource_url).json() + + new_metrics = { 'score': resource_data['score'], 'comments': resource_data['descendants'] } + + # Monitoring : Github + elif monitoring_type == 'github': + resource_id = '/'.join(args.url.split('github.com/')[1].split('/')[0:2]) + resource_url = f"https://api.github.com/repos/{resource_id}" + + headers = {} + if args.token: + headers['Authorization'] = f'Bearer {args.token}' + + resource_data = requests.get(resource_url, headers=headers).json() + + if 'id' not in resource_data: + print_error_github() + exit(1) + + new_metrics = { 'stars': resource_data['stargazers_count'], 'forks': resource_data['forks_count'], 'issues': resource_data['open_issues_count'] } + + # Guard : Initialize metrics on first run + if len(previous_metrics) == 0: + previous_metrics = copy.deepcopy(new_metrics) + continue + + # Determine whether a notification should be sent + for k in new_metrics: + if previous_metrics[k] != new_metrics[k]: + should_notify = True + break + + # Notification + if should_notify: + print_log('Monitoring : New activity', args.quiet) + previous_metrics = copy.deepcopy(new_metrics) + + lines = [ f"{k.capitalize()}: {v}" for k, v in previous_metrics.items() ] + message = "\n".join(lines) + + await notifier.send( + title=args.title, + message=message, + sound=DEFAULT_SOUND if not args.silent else None, + on_clicked=lambda: webbrowser.open(args.url, new=0, autoraise=True) + ) + + await asyncio.sleep(args.frequency) + + print_log('Monitoring : End', args.quiet) + +# Specific setup to handle click on notifications on Mac OS +if platform.system() == 'Darwin': + asyncio.set_event_loop_policy(EventLoopPolicy()) + +asyncio.run(main()) diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..5f1b357 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,2 @@ +requests +desktop_notifier diff --git a/scripts/local-install.sh b/scripts/local-install.sh new file mode 100755 index 0000000..e53948a --- /dev/null +++ b/scripts/local-install.sh @@ -0,0 +1,32 @@ +#!/bin/bash + +# Detect Python and Pip +which python3 +if [ $? -ne 0 ]; then + echo "It seems that Python 3.X is not installed on your system." + echo "Please first ensure that the python3 executable is available" + exit +fi + +which pip3 +if [ $? -ne 0 ]; then + echo "It seems that pip 3.X is not installed on your system." + echo "Please first ensure that the pip3 executable is available" + exit +fi + +# Install dependencies +pip3 install -r requirements.txt +rm requirements.txt + +DESTINATION="/usr/bin" +if [ -d "/usr/local/bin" ]; then + DESTINATION="/usr/local/bin" +fi + +# Remove any previous installation +rm -f $DESTINATION/dawson + +# Install the app's binary +mv dawson $DESTINATION/ +chmod 755 $DESTINATION/dawson diff --git a/scripts/remote-install.sh b/scripts/remote-install.sh new file mode 100755 index 0000000..09900d6 --- /dev/null +++ b/scripts/remote-install.sh @@ -0,0 +1,37 @@ +#!/bin/bash + +# Detect Python and Pip +which python3 +if [ $? -ne 0 ]; then + echo "It seems that Python 3.X is not installed on your system." + echo "Please first ensure that the python3 executable is available" + exit +fi + +which pip3 +if [ $? -ne 0 ]; then + echo "It seems that pip 3.X is not installed on your system." + echo "Please first ensure that the pip3 executable is available" + exit +fi + +# Install dependencies +curl -L -s https://raw.githubusercontent.com/will-moss/dawson/master/requirements.txt > requirements.txt +pip3 install -r requirements.txt +rm requirements.txt + +# Prepare the download URL +GITHUB_LATEST_VERSION=$(curl -L -s -H 'Accept: application/json' https://github.com/will-moss/dawson/releases/latest | sed -e 's/.*"tag_name":"\([^"]*\)".*/\1/') +GITHUB_FILE="dawson" +GITHUB_URL="https://github.com/will-moss/dawson/releases/download/${GITHUB_LATEST_VERSION}/${GITHUB_FILE}" + +# Install/Update the local binary +curl -L -o dawson $GITHUB_URL + +DESTINATION="/usr/bin" +if [ -d "/usr/local/bin" ]; then + DESTINATION="/usr/local/bin" +fi + +mv dawson $DESTINATION +chmod 755 $DESTINATION/dawson