What were you doing on February 14th of 2007? On which tasks were you working on that very day you met your girl friend? When was the last appointments with your dentist? Who called you on telephone during that meeting with your customer last month?

Most people can not answer such questions. With Memacs you can!

Memacs extracts metadata (subjects, timestamps, contact information, …) from many different existing data sources (file names, emails, tweets, bookmarks, …) on your computer and generates files which are readable by GNU Emacs with Org mode.

Example:

   emails              -> memacs-maildir.py  \
   firefox history     -> memacs-firefox.py   |
   SMS                 -> memacs-sms.py       |
   RSS-feeds           -> memacs_rss.py       |
   bank statements     -> memacs-easybank.py  |>  Memacs
   postings            -> memacs-slrn.org     |
   git repository logs -> memacs_git.py       |
   svn repository logs -> memacs_svn.py      /
   |_________________|   |_________________|     |______|
   your personal data      Memacs modules        Org mode

Memacs - as the central component of the system - is a hub for all the connectors that add data from individual data sources. Those connectors are called Memacs modules or short module.

Your agenda automatically gets populated with entries similar to the example shown in the screenshot from the white paper:

You can see some demo data using the modulees for SMS, filenamedatestamps, Twitter, RSS, imap, phonecalls, and git.

• target group: Users of Emacs and Org mode
• skills necessary: creating/modifying shell script code; creating scheduled tasks
• project hosted on https://github.com/novoid/Memacs

Table of Contents

• Installation
• Getting started
• Workflows
• Memacs modules
• Changelog
• Example story
• Background
• Contribute
• License

Installation

Install using Pip

The easiest way of installing Memacs is by using `pip`:

pip install memacs[all]

This gets you the whole set of modules with all dependencies on your system.

If you would like to define the dependencies more fine-grained, you can use one or more of the extras: gps, rss, ical, lastfm, battery, or twitter like the following example:

pip install memacs[gps,rss]

Manual Setup from Git

In case you prefer to set the environment “the old way”, you can do it manually:

• clone or download repository from github
• make sure to use Python version 3
• create a virtualenv or
• export/set PYTHONPATH

PYTHONPATH=/path/to/memacs

• install dependencies

pip install -r requirements.txt

Getting Started

The basic concept of using a Memacs module is following:

1. Choose a Memacs module you want to use and set it up:
   • Read the module descripion files in the docs folder.
   • Each module shares global options but also comes with its very own set of custom arguments.
   • Manually start the module of your choice from the bin folder of Memacs with the argument --help, e.g. bin/memacs_csv.py --help in your command line in order to get an overview of the arguments of this module.
   • You probably want to develop and test a script (=.bat= or =.sh=) for the module invocation in the command line and …
2. When this script works as expected, set up a periodical invocation so that your data gets converted to Org mode via Memacs regularily.
   • This is usually done via operating system dependent methods:
     • Windows: Task Scheduler
     • GNU/Linux: cron job or via systemd
     • macOS: launchd
3. For an update from a running instance of Emacs, independent of a recurrent schedule (e.g., Memacs’ photo module after returning from an excursion), you may add a key binding to Emacs, e.g. C-c m p to your configuration

      (defun mp-update-memacs-photos ()
        "An extra (i.e., not cron-scheduled) run of Memacs' photo module."
        (interactive)
   	 (shell-command
   	   (format "~/org/update-memacs-photos.sh")
   	 )
      )
   
      (global-set-key (kbd "C-c m p") 'mp-update-memacs-photos)
       

   The script called requires the provision of the executable bit. Additional background of this technique is compiled by Mickey Petersen in Mastering Key Bindings in Emacs.

4. Think of another Memacs module you might want to try ;-)

Please make sure you also read the FAQ’s and best practices, as it contains many tips and tricks on how to meet your requirements and on how to choose a reasonable setup.

Workflows

Here are some workflows just to give you an initial impression how Memacs may give you much for digital fun.

The Basics

When one or more Memacs modules are set-up and data starts flowing in to your Memacs Org mode files, you have many options to use Memacs for your personal workflows.

The most basic thing that changes with Memacs is that you might want to see Memacs-processed data in your agenda:

Either you are generating (1) Org mode files that are within your agenda files or you are generating (2) archive files whose more or less empty Org mode file stub is within your agenda files (as described in the FAQs).

This way, the generated time-stamp information gets into your normal agenda (1) or in the extended in archives-mode (by pressing v A in your agenda) that also shows content of the archive files (2).

Filename Time-Stamp and Friends

A Memacs feature I personally use all the time provides a somewhat more complex workflow whose basic data is provided by the filename time-stamp module. The module indexes all my files that contain ISO-datestamps or ISO-timestamps in their file names just like updatedb does for “locate”.

The fun starts, when this module is set-up and the file index is generated (nightly). With the elisp snippets mentioned in the module documentation, I am able to link to any of those files just by specifying their file name. This links never get broken since it does not matter in which folder files are located in any more.

Please refer to my Emacs configuration and look out for all occurrences of tsfile (time-stamp file) which is my custom link for those files.

I even extended this workflow such that my blog system is able to link to tsfile: images independent of their location. Awesome stuff.

Memacs Modules

• arbtt
• battery
• csv
• filenametimestamps
• firefox_history
• chrome_history
• git
• gpx
• ical
• imap
• lastfm
• mu
• phonecalls
• photos
• rss
• simplephonelogs
• sms
• svn
• twitter
• whatsapp
• KODI logs

This module is an example for developers: docs/memacs_csv.org

Those modules are not adopted to the new unified framework yet:

• maildir
• mbox (also useful for newsgroups)
  • see also tmp/emails/mbox/works-for-me-hack/memacs-mbox.org
• bank account:
  • easybank.at: see tmp/bank_statements/easybank.at/memacs-easybank.org
• newsgroups
  • slrn: see tmp/emails/mbox/works-for-me-hack/memacs-mbox.org

Following modules exist as a rough idea only and might get implemented some day (by you?):

• tagstore: see tmp/tagstore/memacs-tagstore.org
• calendar:
  • JPilot-datebook: see tmp/calendars/memacs-jpilot-datebook.org
• tasks:
  • JPilot-todos: see tmp/tasklists/jpilot-todos/memacs-jpilot-todos.org
• blog_systems:
  • Serendipity: see tmp/blog_systems/serendipity/memacs-serendipity.org

Changelog

• Version 2020.06.05.1
  • Features
    • New module: memacs_kodi
    • Andrea Ghensi provided an improved pip setup method
  • Bugfixes
    • fix chrome.py output to include url and title for org headings
    • many improvements for the Chrome module
    • filenametimestamps: fixed check_datestamp_correctness (#97)
• Version 2019.11.06.1
  • Features
    • orgformat is now an external library
  • Bugfixes
    • Fix problems when locale is not en_US
• Versions prior to 2019-10-09.1 are not documented using this changelog. Please read the git commit messages.

Example Story

Imagine you are already using Memacs.

When remembering that day, when you joined an interesting talk about «Getting Things Done» (GTD), you start up your GNU Emacs with your main Org mode file. There you go to the Agenda-view and select this specific day a couple of months ago.

There it is, from 2pm to 3pm you scheduled this talk in your calendar. And then you realize that within this time frame, there appear some JPEG files containing an ISO 8601 time stamp[1] are indexed by one module. (filenametimestamp-module)

This image contains a slide you found interesting and which you photographed using your sleek smartphone. Who would remember having taken a picture during a talk?

Ten minutes after the talk you wrote a short message on Twitter where you mentioned useful URLs for your followers. This time those URLs are handy for yourself too! (Twitter-module)

On the evening of that day you see an entry of an incoming email from the author of the talk. Now you remember having had a cool conversation at the end of the talk where he promised you some additional information about that nice little GTD tool on his computer. Great that you got that link to that email too. Without Memacs you would probably never remembered that email again. (Maildir-module)

And then there were some bookmarks you saved this day, almost all related to great ideas you got from the GTD talk. (delicious-module)

This small story shows only a few use cases where different modules combine given data sources and their information to provide an overall view related to an event. Since Org mode has got links, no actual data has to be duplicated (except the meta data extracted by Memacs). Emails, files, bookmarks, and so forth are linked rather than copied.

[1] with periods instead of colons - just because the ancient limitations of Microsoft based file systems; like «2011-02-14T14.35.42 ideas.jpg»

Background

In 1945, Vannevar Bush wrote a famous article «As We May Think» where he develops the idea of having a «memory extender» called Memex. The memex can store all letters, books, and other information which are related to a person.

Besides having foreseen several technologies like hypertext, he defined a device that holds all metadata and data and provides associative trails to access information.

In the last decade of the previous century Microsoft Research had a research program that resulted in MyLifeBits. This software tried to store each information of the user like office documents, screenshots, name of active windows on the desktop computer, and even automatically took photographs (SenseCam). This word did not result in any (open) software product. Bell and Gemmell wrote a book called «Total Recall».

The Memacs project tries to implement the ideas of Vannevar Bush’s vision with open source and open standards. Hence, it’s name «Memacs» is the obvious combination of «GNU Emacs» and «Memex».

Memacs uses GNU Emacs Org mode to visualize and access information extracted by Memacs modules: using tags, time stamps, full text search, and so forth GNU Emacs is able to derive different views. The most important view probably is the Agenda-view where you can see anything that happened during a specific day/week/month according to the time frame selected. But you can derive other views too. For example you can choose to generate a condensed search result using a boolean combination of tags.

Related to Memacs, the project founder developed a research software called tagstore. This system allows users to store (local) files using tags without a hierarchy of folders. As a natural extension, tagstore targets associative access for (local) files. You might want to check out tagstore too. Memacs and tagstore are a very useful combination.

If you do like to know how to efficiently organize digital files in a simple and operating system independent way, read this blog post from Karl. It might give you ideas for your workflows as well.

Karl also wrote a whitepaper on Memacs which describes Memacs from a scientists point of view.

Similar Projects

• In #88, Alex links to a browser extension from WorldBrain called Memex.
• https://github.com/karlicoss/orger has similar goals: converting data into Org mode

Contribute! We are looking for your ideas:

If you want to contribute to this cool project, please fork and contribute or write an additional module!

See docs/FAQs_and_Best_Practices.org for more developing information.

We are sure that there are a lot of cool ideas for other modules out there! This is just the beginning!

Memacs is designed with respect to minimal effort for new modules.

We are using Python PEP8 and Test Driven Development (TDD).

License

Memacs is licensed under the GPLv3 license.