bURL — Native Messaging Application for LinkRemark and orco

This is a helper application for the LinkRemark browser extension (install Firefox addon from addons.mozilla.org or extension from Chrome Web Store, visit project page at GitHub) which goal is more convenient note taking on visited web pages using Org Mode for GNU Emacs. bURL is also a native messaging host for orco — Org Column add-on for Thunderbird mail and news client, this pair allow to see in message list which ones are mentioned in your notes.

Such kind of application is called native messaging host (application, backend) for a browser add-on. It serves as a bridge that allows to run emacsclient (info "(emacs) Invoking emacsclient") and to search URLs in your notes. Browser extension can not do it directly due to security reasons. In respect to capturing notes, bURL can work with org-protocol (info "(org) Protocols") using emacsclient directly, so it is not necessary to setup desktop-wide org-protocol: handler. LinkRemark source code repository has a simple native messaging app, lr_emacsclient.py Python script, that calls emacsclient for org-protocol: URIs representing captures. The advantage of bURL is that with a bit more configuration, browser add-on will show headings with the same URL as the captured page (a frame on it, or the link) and will allow to open a file in Emacs at particular location. Both features are optional. You are free to capture notes using global org-protocol handler or even to copy notes to clipboard. URL lookup works only if a list of files to search is specified.

Warning There is a problem with browsers distributed as snap or flatpack. Such applications works with additional level of isolation in respect to system files, so external application can not be invoked even through native messaging API. Chromium in Ubuntu is shipped as snap package only since Ubuntu-20.04 LTS focal, for Firefox snap is made default option in Ubuntu-21.10 impish, but Firefox is still can be installed as a deb package using apt. Related Firefox bugs:

• Bug 1661935: Snap: cannot install/manage extensions from extensions.gnome.org,
• Bug 1621763: (flatpak) native messaging support missing.

The helper is tested only on Linux. The stage of its development currently is “proof of concept”: wrapper shell script serves as a configuration file, only exact matches of URLs are checked. URL normalization e.g. default index page, stripping of user activity tracking query parameters are only planned.

Install

To capture pages using bURL, server (info "(emacs) Emacs Server"), should be running in Emacs: M-x server-start to try it, (server-start) in config file, or start Emacs in daemon mode (maybe as a user systemd service) as a long term solution. Without Emacs server the backend is still able to check presence of URLs in configured files, but links to particular lines from LinkRemark preview page do not work.

It is necessary to load org-protocol (info "(org) Protocols") e.g. by customizing list of Org packages M-: (customize-variable-other-window 'org-modules) unless you are going to insert captured notes using C-c C-x C-y, org-paste-subtree.

Certainly you need to download the binary or to build it yourself from sources. GitHub project page has the releases section. Programming language for this project is Go. To compile it should be enough to type

make

and burl_backend file will be created.

In a minimal variant you need just to create a native messaging manifest file. Browsers consider it as a permission for a particular add-on to run the described external application. See the following references for more detailed description and precise location of manifest files:

• Firefox: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_manifests, https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_messaging
• Chrome: https://developer.chrome.com/apps/nativeMessaging

You may create manifest file using any text editor (e.g. Emacs) or use a special command line option:

: "$(moznathostsdir:="$HOME/.mozilla/native-messaging-hosts/")"
test -d "$moznathostsdir" || mkdir "$moznathostsdir"
"${bindir:-.}/burl_backend" -manifest-firefox "$moznathostsdir"

It creates a JSON file with the name of the application, path to executable file, and list of extensions that are allowed to run this application. Next open LinkRemark configuration in browser, grant permission for native messaging (requested during installing in Firefox while compatibility with version 78 ESR is maintained) and specify backend name, that is ”io.github.maxnikulin.burl” by default. Google Chrome and Chromium browsers have different search paths for native manifest and another field to specify list of allowed extensions (so -manifest-chrome should be passed). See User Data Directory in Chromium docs for location of profile directory.

You may specify list of files to search whether some URL has been captured already. There is no a dedicated configuration file yet, however the same effect could be achieved by creating of a small shell wrapper that passes list of files to burl_backend binary. Manifest should contain path to the wrapper. It is possible to create the wrapper and the manifest at once. Likely you would like to adjust list of files in the following example:

: "$(orgdir:="$HOME/org")"
: "$(moznathostsdir:="$HOME/.mozilla/native-messaging-hosts/")"
test -d "$moznathostsdir" || mkdir "$moznathostsdir"
"${bindir:-.}/burl_backend" -manifest-firefox "$moznathostsdir" \
    -wrapper ~/.local/bin/burl_wrapper \
    -org "${orgdir}/notes.org" \
    -org "${orgdir}/capture.org"

Pass -backend NAME option to use custom native host name instead of default io.github.maxnikulin.burl, e.g. burl_wrapper. There is no requirement that executable should be in your PATH, so some custom directory (XDG does not specify libexecdir for users) may be better than ~/.local/bin.

The result should be like the following:

~/.mozilla/native-messaging-hosts/io.github.maxnikulin.burl.json

{
  "name": "io.github.maxnikulin.burl",
  "description": "Burl - LinkRemark interface to Emacs",
  "path": "/home/ubuntu/.local/bin/burl_wrapper",
  "type": "stdio",
  "allowed_extensions": [
    "linkremark@maxnikulin.github.io"
  ]
}

Omitting error-handling stuff, wrapper looks like

~/.local/bin/burl_wrapper

#!/bin/sh -eu
exec /home/ubuntu/.local/bin/burl_backend --log - \
  --org /home/ubuntu/org/notes.org \
  --org /home/ubuntu/org/capture.org

Open LinkRemark settings from context menu for the extension toolbar button or from the about:addons page.

• Set “native-messaging” for export method in “Communication Channel” section to pass captures to Emacs using bURL (optional).
• Set name of native messaging backend to “io.github.maxnikulin.burl” (default) or to the name you specified in the native messaging manifest.
• Grant native messaging permission (in Firefox it is requested during installation to keep compatibility with Firefox-78 ESR that does not allow it as an optional permission).

If you have provided list of files, you can try URL lookup form the add-on preview & debug info page. The “Mentions” sections allows to check any URLs (one per line).

Tuning of Emacs

Open Org file at particular line

When a file is opened at particular line by clicking on some link inside the mentions section of the add-on preview page, it may happen that some parent headings are collapsed. I have not realized what is the safe way to unfold necessary outline headings without possible destructive interference with other invocations of emacsclient.

For a while you may alleviate the issue by adding the following peace of code to your Emacs init file

(defun lr-org-reveal-folded-line ()
  (when (derived-mode-p 'org-mode)
    (org-reveal)))

(add-hook 'server-visit-hook #'lr-org-reveal-folded-line)

Alternatively use C-c C-r that is a binding for M-x org-reveal.

Frame (window) for capture or to show existing link

I am unsure what is the best way to create frame (if it does not exist yet) for capture or to show the place where an URL is mentioned. Personally I do not like when Emacs creates new frame in a terminal window just because I have launched a browser from this terminal. So linkremark-ensure-frame function is invoked (if it is defined) before over actions. An example how to it may look like:

 (defun linkremark-ensure-frame ()
   (or (memq 'x (mapcar #'framep (frame-list)))
	(select-frame
	 (make-frame '((name . "LinkRemark") (window-system . x))))))

It may open a file in a frame located on another desktop. I have not figured out how to cook something better from the following ingredients: server-window, display-buffer-alist, frames-on-display-list, default-frame-alist.

WebExtensions package for Go

This project has a simple Go package that provides tools for using net/rpc/jsonrpc (frozen) package from Go standard library through stdin and stdout pipes accordingly to native messaging protocol in browsers. Its advantage is no external dependencies, its disadvantages are absence of support of request contexts and single parameter of RPC methods that must be passed as single-element array. You may consider JSON-RPC 1.0 as not flexible enough in comparison to version 2.0 of the protocol.

See pkg/webextensions folder for more detailed description. The examples directory contains a small backend and a Firefox extension that can call its methods.

License

GPL v3

An example of browser add-on examples/webextensions_addon is licensed under MPL-2.0 (it is no more than a demo for the pkg/webextensions Go package).