Skip to content

Latest commit

 

History

History
113 lines (90 loc) · 4.44 KB

README.md

File metadata and controls

113 lines (90 loc) · 4.44 KB
Raw

Overleaf-Go

Welcome to the Golang port of Overleaf!

This repository is a monorepo for all the backend services and other useful command line utils. The frontend code remains in a fork of the Node.js monorepo.

Highlights compared to the original Node.js implementation

  • The backend code is written in a statically typed language
  • Monolith-first approach, eliminating latency overhead and network chatter -- just run this single binary, and you are good (compiles can still be scaled horizontally, other components as well - but they are self-contained and do not expose internal APIs on the network)
  • Mongodb was replaced with PostgreSQL (an automated data migration exists via cmd/m2pq, you can find a mongodb implementation and edgedb implementation in the git history too)
  • Postgres enables single round-trip data fetching for complex pages like the project dashboard or the project editor page
  • Postgres enables atomic updates to the filetree and strong unique constrains inside the tree
  • No admin capabilities in the web app (command line utils exist instead)
  • Highly optimized compile process (<200ms are possible for incremental compiles, <20ms for synctex aka navigating between code/PDF location, zero content is fetched from Postgres for incremental compiles)
  • Homegrown real-time service with minimal overhead for connection startup (zero server round trips inside the websocket for "joining" a project)
  • JWT based authentication and authorization for many endpoints (epochs in the DB enable invalidation of tokens ahead of their expiry)
  • Highly optimized document updating process (merged redis keys reduce redis calls by an order of magnitude, re-using of already fetched content for processing multiple updates and flushing them all the way to history/DB)
  • Worker-stealing for spellchecker (this allows less popular languages to take over idle worker from a more popular language)
  • Chaining of network proxies for fetching external resources (multiple hops between proxies allow stronger firewalling of services, e.g. hop to local service and then hop to external cloud -- only that local service needs internet access)
  • Stronger CSP and zero inline scripts in HTML templates
  • Minify and compile HTML templates at boot-time in well under 1s
  • Caching for documentation content and images
  • Website translations on a single domain
  • No local filesystem backend for storing binary files (all major providers expose an S3 compatible API today and min.io can be used for self-hosting)

Getting started

There are little to no run-time defaults and strict validation of options is in place for flagging missed/invalid directives in your config.

You can use the config generator which has opinionated options and auto-detection for Docker details (rootful, rootless, Mac).

With Golang installed and docker-compose v2:

go run ./cmd/config-generator | tee .env
docker compose up
Without Golang installed

  • Without Golang installed on Linux and docker-compose v2:

    docker run --rm -v `pwd`:`pwd` -w `pwd` golang:1.22.6 \
  go build -tags osusergo,netgo ./cmd/config-generator
./config-generator | tee .env
docker compose up

  • Without Golang installed on an Intel based Mac and docker-compose v2:

    docker run --rm -v `pwd`:`pwd` -w `pwd` \
  -e GOOS=darwin -e GOARCH=amd64 golang:1.22.6 \
  go build -tags osusergo,netgo ./cmd/config-generator
./config-generator | tee .env
docker compose up

  • Without Golang installed on a M1/M2 based Mac and docker-compose v2:

    docker run --rm -v `pwd`:`pwd` -w `pwd` \
  -e GOOS=darwin -e GOARCH=arm64 golang:1.22.6 \
  go build -tags osusergo,netgo ./cmd/config-generator
./config-generator | tee .env
docker compose up

Note: For more advanced usages (e.g. email delivery), see $ go run ./cmd/config-generator --help or $ ./config-generator --help.

Docker-compose will build all the images, binaries and frontend assets, run the minio setup, create the postgres schema and finally start it all up!

The first startup may take a while to complete. Watch out for a message that says "Setup finished!".

You can now open http://localhost:8080/register in your favorite browser and create your first account!

Tip: Use $ docker compose up --detach to keep the services running in the background.