Git Repo Preservation Project (GRPP)

Created by TheMitoSan, this is a tool written in ts that aims to help importing / preserving git repos.

Important

GRPP is a tool that only focus on git repos itself.

If you want to preserve GitHub repos including issues, releases and more - check out jan9103's github-repo-backuper.

Table of contents

• Requirements
• How to build
  • Easy mode
  • Development mode
  • Production mode
• How to uninstall
• How to use
  • How to import git repos
    • Import single repo
    • Import multiple repos
  • Get all repos from a specific user
  • How to update repos
    • Update a single repo
    • Update all repos
• External tools used on this project
• External code snippets used on this project

Requirements

• git, npm and latest node version installed.

You can install all required packages by running some of the commands below:

# Arch based distros <3
sudo pacman -Sy nodejs git

# Ubuntu based distros
sudo apt update
sudo apt install nodejs git

# Fedora based distros
sudo dnf update
sudo dnf install nodejs git

How to build

There are three ways of compiling GRPP: Easy mode, development mode or production mode.

Easy mode

If you just want GRPP up and running, just run the following command:

npm i && npm run bi

This will install all required packages, compile the project and install GRPP system-wide.

Development mode

In order to compile, run the following commands:

npm i && npm run webpack

The compiled script will be on Build dir.

TIP: You can also run npm run dev - ts will watch all changes on your code and will compile script on the go! In this case, the compiled script will be on App dir instead.

In order to run GRPP, you will need to run like this:

# webpack
node Build/grpp.js

# OR

# Dev mode
node App/grpp.js

Production mode

First, you will need to install all dependencies and compile the project.

To do this, run the following commands:

npm i && npm run build

This will generate a .tgz file inside Build dir. In order to install, run the following command:

sudo npm i -g grpp.tgz

Important

You MUST run this command as sudo, since it will install grpp as a global package.

If everything is fine, you will be able to call grpp system-wide.

How to uninstall

It's simple! Just run the following command:

sudo npm remove grpp

How to use

Important

To know everything GRPP can do, we recommend checking out the command list.

The first thing you need to do is initialize a folder, where it will store it's settings file and all repos.

To do this, run the following command:

grpp --init

After initializing, you will be able to start importing git repos.

How to import git repos

There is some ways you can do this: By importing them one by one, a repo list or even all repos from a specific user.

Single repo

To do that, you can use the --import= command with your desired git url:

grpp --import=http://github.com/themitosan/themitosan.git

If everything is fine, you will be able to import this git repo to your current location organized by host/user/repo.

(On this case, themitosan.git will be located on YOUR_CURRENT_PATH/repos/github.com/themitosan/themitosan.git)

Multiple repos

You can import a list of repos from a text file! In order to do this, you can use the following command:

grpp --importList=YOUR_GIT_REPOS_LIST.txt

Doing that, all repos on YOUR_GIT_REPOS_LIST.txt will be imported to your current path.

Important

Make sure there is only one git url per line!

Getting all repos from a specifc user

Since common git hosting websites (like GitHub or GitLab) API's allows fetching user data, GRPP is capable to get all repos from a selected list of users and importing them!

In order to do that, you can use the following command:

grpp --getReposFrom=USERNAME

You will be prompted asking where GRPP should seek USERNAME repos.

After selecting one of selected options, it will display all repos from the provided user and asking if you want to import.

Important

Since some git hosting websites limits each ip that fetches data very often, GRPP is configured to fetch only 500 repos per user (100 repos per fetch).

If you want to start fetching data from a specific page or fetch a specific number of pages, you can use these commands below:

# Fetch 10 repo pages (Default: 5)
grpp --setMaxFetchPages=10 --getReposFrom=themitosan

# Set start fetch page to 4 (Default: 1)
grpp --setStartPage=4 --getReposFrom=themitosan

How to update repos

There is two ways of doing this: by updating a specifc repo or by updating all at same time.

Update a single repo

To update a single repo, just run the following command:

grpp --update=[PATH_TO_GRPP_REPO]

Update all repos

This is where GRPP shines! To update all repos, run the following command:

grpp --updateAll

This will create a determinated number of processes, updating all repos that can be updated on GRPP.

To set the number of processes running, you will need to divide the number of repos that will be updated per process.

Example: Let's just say that you have 100 repos on your database. If you want 4 processes, you can run the following command:

grpp --maxReposPerList=25 --saveSettings

This will create four processes, each updating 25 repos.

After processing all repos, GRPP will show the update results and save a copy on Logs dir.

External tools used on this project

• Webpack
• Photopea

External code snippets

Some code snippets from internet were used on this project. You can see all of them listed below:

• Console text color sheet
• Original INI Parser
• Original DNS internet check
• Original snippet to format time in hhmmss
• Original snippet to convert array in smaller chunks
• Original snippet that clears console + history

^{Some parts of this software derivates from TMS Engine source code.}