Re-write Inkpath with OpenCV (#22)

* Re-write in C++ 

* upsampling

* autotrace doesn't like images that come frm opencv

* chom

* a

* I wanna try this with opencv

* Cleanup and usability

* Path parsing actually works™, Not super useful for the actual tool, but good for debugging lol

* Contours

* Why not just re-write the whole thing in c++

* Update build instructions 1/?

* Clean up ipcv_obj and Makefile

* Update documentation

.gitignore

@@ -6,3 +6,5 @@ build/
 *.svg
 do.sh
 ImageTranscription/inkpath.so
+ImageTranscription/ipcvobj.so
+*.gch

HACKING/Dockerfile

@@ -1,21 +1,21 @@
 FROM docker.io/debian
 
-RUN touch /root/.Xauthority && \
-  apt-get update && \
-  apt-get install -y tmux git gdb vim x11-apps #&& \
-#  apt-get install -y build-essential pkg-config autoconf intltool autopoint libtool libglib2.0-dev build-essential libmagickcore-dev libpstoedit-dev imagemagick pstoedit && \
-#  apt-get install -y cmake libgtk-3-dev libpoppler-glib-dev portaudio19-dev libsndfile-dev \
-#  dvipng texlive libxml2-dev liblua5.3-dev libzip-dev librsvg2-dev gettext lua-lgi
+RUN touch /root/.Xauthority
+RUN apt-get update
 
-# Disabled for testing
-#RUN cd && git clone https://github.com/yy502/autotrace && \
-#  cd autotrace && ./autogen.sh && LD_LIBRARY_PATH=/usr/local/lib ./configure --prefix=/usr && make && make install
+# Line 10: Inkpath
+# Line 11: Xournalpp
+# Line 13: debugging tools
+RUN apt-get install -y make liblua5.4-dev build-essential pkg-config libglib2.0-dev libpng-dev && \
+  apt-get install -y cmake libgtk-3-dev libpoppler-glib-dev portaudio19-dev libsndfile-dev \
+  dvipng texlive libxml2-dev libzip-dev librsvg2-dev gettext lua-lgi \
+  libgtksourceview-4-dev
+RUN apt-get install -y tmux git gdb vim x11-apps tree wget unzip
 
-# Mount these yourself.
-#git clone https://github.com/xournalpp/xournalpp && \
-#git clone https://gitub.com/willnilges/inkpath && \
-#RUN cd && git clone https://github.com/willnilges/xournalpp && \
-#cd xournalpp && mkdir build && cd build && cmake .. && cmake --build .
-#
-#RUN cd && git clone https://github.com/willnilges/inkpath && \
-#cd inkpath && make lua-plugin && cp -r ImageTranscription ../xournalpp/plugins && cp ImageTranscription/inkpath.so ../xournalpp/build/src
+# Build and install OpenCV into the container
+COPY ./build-opencv.sh .
+RUN bash /build-opencv.sh 48
+
+# Build and install Xournal++
+COPY ./build-xopp.sh .
+RUN bash /build-xopp.sh 48

HACKING/README.md

@@ -1,28 +1,28 @@
+# Thanks for contributing :)
 
-This Dockerfile should have everything you need (sans Xforwarding, TODO) to make a development container.
+## Setting up the development environment
 
-TODO: Not sure if I want these as separate layers. It makes sense to me at 01:36, though.
+This guide will walk you through everything you need to work on this project, including building OpenCV and Xournalpp.
 
-We got:
-1. Packages required for the container
-2. Packages to build AutoTrace
-3. Packages to build Xournalpp and Inkpath
-(Some of these overlap)
+(You should have all relevant development files (including this repo) located at `~/Code/xopp-dev`. This can be configured in `launch-environment.sh` by the `$CODE_PATH` variable)
 
-Notes: imagemagick, pstoedit are optional.
-
-You'll need to `xauth add <SECRET>` to make Xforwarding work. Find it with `xauth list` on your host.
-
-Run this with:
+- First, build the Dockerfile. This will download all the packages you need to compile inkpath and friends. OpenCV is the backend that now powers this project, and you might want Xournalpp as source code so you can debug more easily.
+```
+./HACKING/build-environment.sh
+```
 
+- Next, launch it. This container will do Xforwarding for you so that you can run Xournalpp on your desktop and do development. All subsequent build stuff should happen in here.
 ```
-podman build . --tag xopp-dev
-podman run --name=xopp-dev -dit -e DISPLAY=$DISPLAY --network=host --cap-add=SYS_PTRACE -v /home/$USER/Code/inkpath_dev:/mnt/inkpath_dev -v /tmp/.X11-unix:/tmp/.X11-unix xopp-dev
+./HACKING/launch-environment.sh
 ```
 
-Or use the provided script.
+- If you did it right, you should have an `xopp-dev` directory in your container.
+```
+cd /xopp-dev/inkpath/
+make dev-install
+```
 
-## Debugging the Lua
+## Debugging Lua
 
 You might find the following to be useful
 - https://github.com/kikito/inspect.lua

HACKING/build-environment.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+set -e
+cd HACKING
+podman build . --tag xopp-dev

HACKING/build-opencv.sh

@@ -0,0 +1,27 @@
+#!/bin/bash
+
+set -e
+
+cores=4
+
+# For compiling faster
+if [ -n "$1" ]; then
+    cores=$1
+fi
+
+# Create exterior directories
+mkdir -p ../opencv
+cd ../opencv
+
+wget -O opencv.zip https://github.com/opencv/opencv/archive/4.5.1.zip
+
+unzip opencv.zip
+# Create opencv build directory
+mkdir -p build && cd build
+# Configure
+cmake  ../opencv-4.5.1 -DBUILD_SHARED_LIBS=OFF -DOPENCV_GENERATE_PKGCONFIG=YES -DWITH_GTK=OFF
+# Build
+cmake --build . -j$cores
+
+make install
+

HACKING/build-xopp.sh

@@ -1,8 +1,12 @@
 #!/bin/bash
 
+# It's assumed that xournalpp will be cloned outside this repo :)
+cd ..
+git clone https://github.com/xournalpp/xournalpp
+
 if [[ "$1" == "clean" ]]; then
     echo 'Deleting build dir'
-    rm -rf ../../xournalpp/build
+    rm -rf ../xournalpp/build
     exit 0
 fi
 
@@ -13,8 +17,7 @@ if [ -n "$1" ]; then
     cores=$1
 fi
 
-
-cd ../../xournalpp
+cd ../xournalpp
 mkdir build
 cd build
 cmake ..

HACKING/launch-environment.sh

@@ -36,7 +36,8 @@ podman run --name=xopp-dev --rm -it                   \
     --cap-add=SYS_PTRACE                              \
     --group-add keep-groups                           \
     --annotation io.crun.keep_original_groups=1       \
-    -v "$xauth_path"/.Xauthority:/root/.Xauthority:Z \
+    -v "$xauth_path"/.Xauthority:/root/.Xauthority:Z  \
     -v "$CODE_PATH":/xopp-dev:Z                       \
+    -v /scratch/wilnil:/scratch:Z                     \
     -v /tmp/.X11-unix:/tmp/.X11-unix xopp-dev
 rm -rf "$xauth_path"

ImageTranscription/main.lua

@@ -6,31 +6,29 @@ end
 
 -- Callback if the menu item is executed
 function drawStroke()
-    inkpath = require 'inkpath'
-    -- path = app.getFilePath({'*.ppm', '*.png', '*.pbm', '*.pnm', '*.bmp', '*.tga', '*.yuv', '*.pgm', '*.gf'}) -- Autotrace 0.40.0 supports ppm, png, pbm, pnm, bmp, tga, yuv, pgm, gf
-    path = app.getFilePath({'*.png', '*.bmp'}) -- The current version of Autotrace I'm using only supports PNGs.
-    image_scale = app.msgbox("Select tracing scale", {[1] = "Small", [2] = "Medium", [3] = "Large"})
-    output = inkpath.transcribe_image(path, image_scale)
+    print("Inkpath Activated. Transcribing image....")
+    inkpath = require "ipcvobj"
+    path = app.getFilePath({'*.jpg', '*.png', '*.bmp'}) -- The current version of Autotrace I'm using only supports PNGs.
+    --image_scale = app.msgbox("Select tracing scale", {[1] = "Small", [2] = "Medium", [3] = "Large"}) -- TODO: implement this again.
+    image_scale = 1
+    scaling_factor = 10.0 -- THIS IS A NEW THING! HOW MUCH DO YOU WANT TO DIVIDE YOUR SHIT BY!? MUST BE FLOAT!
+    local obj = IPCVObj(path, 1)
     print("Strokes retrieved.")
-    strokes = {}
-    single_stroke = {}
-    for key, value in pairs(output) do
-        if value[1] == -1.0 and value[2] == -1.0 then -- If we get a delimiting pair, add our current stroke to the stroke table.
-            table.insert(strokes, {
-                ["coordinates"] = single_stroke,
-            });
-            single_stroke = {}
-        else
-            table.insert(single_stroke, value[1]) -- Y coord
-            table.insert(single_stroke, value[2]) -- X coord
-        end
+    contourCt = obj:getLength()
+    print("Got ", contourCt, " strokes.")
+    -- TODO: This could be much, MUCH faster.
+    for i = 0,contourCt-1,1 do
+        pointCt = obj:getContourLength(i)
+        x_points, y_points = obj:getContour(i, scaling_factor)
+        app.addStrokes({
+            ["strokes"] = {
+                {
+                    ["x"] = x_points,
+                    ["y"] = y_points,
+                },
+            },
+            ["allowUndoRedoAction"] = "grouped",
+        })
     end
-    -- When we've assembled our table of strokes, call the addSplines function
-    -- Not going to pass any options since I want to use the current tool options.
-    app.addSplines({
-        ["splines"] = strokes,
-        ["allowUndoRedoAction"] = "grouped",
-    })
-    app.refreshPage()
-    print("done")
+    print("Image Transcription Complete. Exiting Inkpath.")
 end

Makefile

@@ -5,49 +5,67 @@ WARNINGS = -Wall -Wextra -Wpedantic -Wconversion -Wformat=2 -Winit-self \
 	-Wmissing-include-dirs -Wformat-nonliteral -Wnested-externs \
 	-Wno-unused-parameter -Wold-style-definition -Wredundant-decls -Wshadow \
 	-Wstrict-prototypes -Wwrite-strings
-#LIGHT_WARNINGS = -Wall
+LIGHT_WARNINGS = -Wall
 PLUGIN_NAME=ImageTranscription
+SO_NAME=ipcvobj.so
 LUA_VERSION=5.4
 SO_INSTALL_PATH=/usr/lib64/lua/$(LUA_VERSION)# Just one of many possible destinations :)
-
-# CFLAGS += -std=gnu99
+XOPP_DEV_INSTALL_PATH=/xournalpp
 
 .PHONY: clean install uninstall dev-install dev-uninstall
 
-# TODO: `-g` is for debugging. Make a target that supports debugging separately from primary compilation
+ip_source := $(wildcard src/ipcv_obj/*.cpp)
+cv_source := $(wildcard src/cv/*.cpp)
 
-at_source := $(wildcard src/autotrace/*.c src/autotrace/*.h)
+lua_deps=`pkg-config --cflags --libs lua`
+cv_deps=`pkg-config --cflags --libs --static opencv4`
 
-lua-plugin: src/lua_util.c $(at_source)
-	mkdir -p build
-	$(CC) $(LIGHT_WARNINGS) $(CFLAGS) src/lua_util.c $(at_source) -g `pkg-config --cflags --libs lua glib-2.0` -fPIC -shared -o $(PLUGIN_NAME)/inkpath.so
+.PHONY: build_dir
+build_dir:
+	@mkdir -p build
+
+# Compiles and statically links Inkpath's OpenCV code to the necessary OpenCV libraries
+ipcv: $(cv_source)
+	@mkdir -p build
+	g++ -c $(cv_source) $(lua_deps) $(cv_deps) -fPIC -static
+	@mv *.o build
+	ar -crsT build/libipcv.a build/*.o
 
+# Compiles Inkpath's shared object library
+lua-plugin: $(ip_source) ipcv
+	g++ $(LIGHT_WARNINGS) $(ip_source) -L/xopp-dev/inkpath/build -lipcv $(cv_deps) $(lua_deps) -g -fPIC -shared -o $(PLUGIN_NAME)/$(SO_NAME)
+
+# Installs the plugin into your Xournalpp installation
 install: lua-plugin
 	cp -r $(PLUGIN_NAME) /usr/share/xournalpp/plugins/
 	mkdir -p $(SO_INSTALL_PATH)
-	cp -r $(PLUGIN_NAME)/inkpath.so $(SO_INSTALL_PATH)/inkpath.so
+	cp -r $(PLUGIN_NAME)/$(SO_NAME) $(SO_INSTALL_PATH)/$(SO_NAME)
 
+# Remove the plugin files from the xournalpp install dir
 uninstall:
 	rm -rf /usr/share/xournalpp/plugins/$(PLUGIN_NAME)
-	rm $(SO_INSTALL_PATH)/inkpath.so
+	rm $(SO_INSTALL_PATH)/$(SO_NAME)
 
 # Used to install the plugin into a source code repository of xournalpp
-dev-install:
-	cp -r $(PLUGIN_NAME) ../xournalpp/plugins
-	cp -r HACKING/StrokeTest ../xournalpp/plugins
-	cp $(PLUGIN_NAME)/inkpath.so ../xournalpp/build/
+dev-install: lua-plugin
+	cp -r $(PLUGIN_NAME) $(XOPP_DEV_INSTALL_PATH)/plugins
+	cp -r HACKING/StrokeTest $(XOPP_DEV_INSTALL_PATH)/plugins
+	cp $(PLUGIN_NAME)/$(SO_NAME) $(XOPP_DEV_INSTALL_PATH)/build/
 
+# Remove the plugin from the development environment
 dev-uninstall:
-	rm -rf ../xournalpp/plugins/$(PLUGIN_NAME)
-	rm -rf HACKING/StrokeTest ../xournalpp/plugins
-	rm ../xournalpp/build/inkpath.so
+	rm -rf $(XOPP_DEV_INSTALL_PATH)/plugins/$(PLUGIN_NAME)
+	rm -rf HACKING/StrokeTest $(XOPP_DEV_INSTALL_PATH)/plugins
+	rm $(XOPP_DEV_INSTALL_PATH)/$(SO_NAME)
 
-stroketest:
-	cp -r HACKING/StrokeTest ../xournalpp/plugins
+# For generating a CV debugging binary
+ipcv-debug: $(cv_source) 
+	mkdir -p build
+	g++ src/cv/debug/debug.cpp $(cv_source) `pkg-config --cflags --libs --static opencv4` -static -o build/ipcv
 
-stroketest-uninstall:
-	rm -rf HACKING/StrokeTest ../xournalpp/plugins
+help:
+	@echo ipcv lua-plugin install uninstall dev-install dev-uninstall ipcv-debug
 
 clean:
 	rm -rf build
-	rm $(PLUGIN_NAME)/inkpath.so
+	rm $(PLUGIN_NAME)/$(SO_NAME)

README.md

@@ -21,15 +21,23 @@ Inkpath is a project designed to crunch those whiteboard photos into easily
 editable [Xournalpp](https://github.com/xournalpp) note files so that you can
 drop your whiteboard scrawlings directly into your lecture notes. Convenient!
 
-This uses [autotrace](https://github.com/autotrace/autotrace) to translate
-whiteboard markings into splines. Those splines are then passed directly to the
-Xournal++ API and rasterized, before being placed onto the working layer.
+The project consists of a lua script and a shared object library written in C++
+statically linked with some OpenCV utils. The project now uses OpenCV in place
+of Autotrace to perform an Otsu Threshold. The thresholded image is then inverted,
+and skeletonized, producing the centerline of each individual object in the image
+(which at this point should be just markings). The resulting image is then scanned
+for contours. These contours are pushed onto the lua stack, and passed to the 
+Xournal++ API. Unlike the previous implementation, this operates purely on rasters.
 
 ## Installation and Usage
 
-When you run `make lua-plugin`, it will compile inkpath and place `inkpath.so` in the ImageTranscription directory.`make install` will copy that directory to your Xournalpp plugins folder, and Inkpath will be installed. You can then use it from the 'Plugins' menu from within Xournalpp.
+### Prerequisites
+
+You'll need to statically compile and install OpenCV (see `HACKING/build-opencv.sh`), then install Xournalpp using your package manager of choice (or compile from source).
+
+_I'm not 100% sure that all the API changes I've made have been released. They are merged, but might not be in your package manager._
 
-_Prerequisite: You must have xournalpp installed via your package manager of choice_
+### Installation
 
 1. Download dependencies
 
@@ -50,10 +58,13 @@ pacman -S base-devel pkg-config lua libpng
 
 2. Compile and install Inkpath
 ```
-# Download and build Inkpath (Deps included in Xournalpp)
 git clone https://github.com/willnilges/inkpath.git
 cd inkpath
 make install
 ```
 
+When you run `make lua-plugin`, it will compile inkpath and place `inkpath.so` in the ImageTranscription directory.`make install` will copy that directory to your Xournalpp plugins folder, and Inkpath will be installed. You can then use it from the 'Plugins' menu from within Xournalpp.
+
+_Inkpath is coming to a package manager near you soon™!_
+
 <img src="https://forthebadge.com/images/badges/works-on-my-machine.svg" alt="C badge" height="30px"/>