Skip to content

Latest commit

 

History

History
228 lines (171 loc) · 7.5 KB

README.rst

File metadata and controls

228 lines (171 loc) · 7.5 KB
Raw

moin2git

A tool to migrate the content of a MoinMoin wiki to a Git backed wiki engine like Gollum, Realms, Waliki or similar.

Install

git clone --recursive https://github.com/mgaitan/moin2git.git
[sudo] pip install -r requirements.txt

If you also want to convert each page to reStructuredPage format, (see --convert-to-rst) you will need to install MoinMoin:

[sudo] pip install moin

Usage

tin@morochita:~$ python moin2git.py --help
moin2git.py

A tool to migrate the content of a MoinMoin wiki to a Git based system
like Waliki, Gollum or similar.

Usage:
  moin2git.py migrate <data_dir> <git_repo> [--convert-to-rst] [--users-file <users_file>]
  moin2git.py users <data_dir>
  moin2git.py attachments <data_dir> <dest_dir>

Arguments:
    data_dir  Path where your MoinMoin content is
    git_repo  Path to the target repo (created if it doesn't exist)
    dest_dir  Path to copy attachments (created if it doesn't exist)

Options:
    --convert-to-rst    After migrate, convert to reStructuredText
    --users-file        Use users_file to map wiki user to git commit author

Workarounds

If you need to convert the markup to rst, you will need a working moinmoin instance. For a fast and dirty configuration, put your data in a directory named wiki, and copy wikiconfig.py in the same level:

wikiconfig.py
wiki/
├── data/

Then copy moin2git/moin2rst/text_x-rst.py to wiki/data/plugins/formatters/

How it works

MoinMoin is a wiki engine powered by Python that store its content (including pages, history of changes and users) as flat files under the directory /data.

An overview of the structure of this tree is this:

data/
├── cache
│   │     ...
│
├── pages
│   │
│   ├── AdoptaUnNewbie
│   │   ├── cache
│   │   │   ├── hitcounts
│   │   │   ├── pagelinks
│   │   │   └── text_html
│   │   ├── current
│   │   ├── edit-lock
│   │   ├── edit-log
│   │   └── revisions
│   │       ├── 00000001
│   │       ├── 00000002
│   │
│   ├── AlejandroJCura
│   │   ├── cache
│   │   │   ├── pagelinks
│   │   │   └── text_html
│   │   ├── current
│   │   ├── edit-lock
│   │   ├── edit-log
│   │   └── revisions
│   │       ├── 00000001
│   │       ├── 00000002
│   │       └── 00000003
│   │
│   ├── AlejandroJCura(2f)ClassDec(c3b3)
│   │   ├── cache
│   │   │   ├── pagelinks
│   │   │   └── text_html
│   │   ├── current
│   │   ├── edit-lock
│   │   ├── edit-log
│   │   └── revisions
│   │       ├── 00000001
│   │       ├── 00000002
│   │       └── 00000003
 ...
│   └── YynubJakyfe
│       ├── edit-lock
│       └── edit-log
│
└── user
    ├── 1137591729.59.35593
    ├── 1137611536.06.62624
    ├── 1138297101.79.62731
    ├── 1138912320.61.21990
    ├── 1138912840.93.11353
    ...

  • Each wiki page (no matter how deep its url be) is stored in a directory /data/pages/<URL>. For example in our example the url /AlejandroJCura/ClassDec%C3%B3 [1] is data/pages/AlejandroJCura(2f)ClassDec(c3b3)

  • The content itself is in the directory /revisions, describing the history of a page. Each file in this directory is a full version of a the page (not a diff).

  • The file /data/pages/<URL>/current works as a pointer to the current revision (in general, the more recent one, but a page could be "restored" to an older revision). For example:

    tin@morochita:~/lab/moin$ cat data/pages/AlejandroJCura/current
00000003

  • The edit-log file describes who, when and (if there is a log a message) why:

    tin@morochita:~/lab/moin$ cat data/pages/AlejandroJCura/edit-log
  1141363609000000    00000001    SAVENEW AlejandroJCura  201.235.8.161   161-8-235-201.fibertel.com.ar   1140672427.37.17771     Una pagina para mi?
  1155690306000000    00000002    SAVE    AlejandroJCura  201.231.181.174 174-181-231-201.fibertel.com.ar 1140672427.37.17771
  1218483772000000    00000003    SAVE    AlejandroJCura  201.250.38.50   201-250-38-50.speedy.com.ar 1140672427.37.17771

    The data logged is (in this order, separated by tabs):

    EDITION_TIMESTAMP, REVISION, ACTION, PAGE, IP, HOST, USER_ID, ATTACHMENTS, LOG_MESSAGE

  • The USER_ID point to a file under the directory /data/user contained a lot of information related to the user. For example:

    (preciosa)tin@morochita:~/lab/moin$ cat data/user/1140549890.71.33402
remember_me=1
theme_name=pyar
editor_default=text
show_page_trail=1
disabled=0
quicklinks[]=Noticias
css_url=
edit_rows=20
show_nonexist_qm=0
show_fancy_diff=1
tz_offset=-10800
subscribed_pages[]=
aliasname=
remember_last_visit=0
enc_password={SHA}5kXNi+HjaTCGItkg6yTPNRtSDGE=
email=mautuc@yahoo(....)
show_topbottom=0
editor_ui=freechoice
datetime_fmt=
want_trivial=0
last_saved=1219176737.74
wikiname_add_spaces=0
name=MauricioFerrari
language=
show_toolbar=1
edit_on_doubleclick=0
date_fmt=
mailto_author=0
bookmarks{}=

Solving the puzzle

moin2git.py uses git (via the wonderful sh) to handle the history, so don't need multiples files to track differents revision of a page

For instance, in the root of our target directory (the git repo) we should get a file AlejandroJCura:

  • 3 revisions (commits), from revisions/00000001 until revisions/00000003
  • the author name/nickname and email (if available) is parsed from the user file of each revision. To know who and when made what version, moin2git.py parses the edit-log file of each page.

We should also get a file AlejandroJCura/ClassDecó [2] where, in this case, AlejandroJCura/ is a directory.

Commit authors

The option --users-file acepts a file that will be used to map wiki users to git commit authors.

The output of the command moin2git.py users <data_dir> can be used as input. For each users the required fields are name and email.

[1]http://python.org.ar/AlejandroJCura/ClassDec%C3%B3
[2]Note we should parse the ugly escaping. (2f) is / and determines the left part is a directory. (c3b3) means %C3%B3, i.e. ó