Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

credentials: Use cross-platform credentials.toml instead of Unix-specific .env files. #2

Merged
merged 22 commits into from
Mar 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .env.sample

This file was deleted.

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,7 @@ vls.log

# Enviroment files
*.env
*.toml

# JetBrains Files
.idea
116 changes: 72 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
<!--suppress HtmlDeprecatedAttribute -->
<div align="center">
<h1><em>klonol</em></h1>

[vlang.io](https://vlang.io) | [hungrybluedev](https://hungrybluedev.in/)

</div>
<!--suppress HtmlDeprecatedAttribute -->
<div align="center">

[![CI][workflowbadge]][workflowurl]
[![License: MIT][licensebadge]][licenseurl]
[![Git Latest Tag][gittagbadge]][gittagurl]
[![CI][workflow_badge]][workflow_url]
[![License: MIT][license_badge]][license_url]
[![Git Latest Tag][git_tag_badge]][git_tag_url]

</div>

Expand All @@ -23,13 +25,15 @@ authenticated user).
to the authenticated user.
3. You can list all available repositories, clone them, or run `git pull`
on existing clones. It uses the user's SSH key to clone.
4. Cross-platform! We've switched to using TOML for storing credentials and
have tested this project extensively on Windows as well.

## Motivation

I self-host my Gitea instance. It contains several private repositories. There
are some instances where I need to upgrade the server or perform some
maintenance. Although I use [Docker Compose](https://docs.docker.com/compose/)
with mounted volumes to manage the Gitea instance, there may be times where
with mounted volumes to manage the Gitea instance, there may be times when
data retention is not possible. I need to restart my service from scratch. In
order to help with these "from-scratch" scenarios, I wrote this tool.

Expand All @@ -45,7 +49,8 @@ git server in peace.
2. You must have a GitHub or Gitea account.
3. [Add an SSH key](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account)
to your appropriate account.
4. Generate an [_personal access token_](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
4. Generate a [_personal access
token_](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
with the minimum scope of `repo` (to allow viewing private repositories)
and set an expiration of 7 days or the lowest possible. Regenerate this
key when it expires.
Expand Down Expand Up @@ -76,6 +81,10 @@ v build.vsh

The optimized `klonol` binary is saved in the `bin` subdirectory.

**Note:** If you are working on _klonol_ locally and want faster builds for quicker
prototyping, use `v build.vsh -fast` to use TCC and build an unoptimised executable
quickly.

**Step 3:** Add to `PATH`

Add the `bin` subdirectory to your `PATH`. You can edit your `~/.bashrc` file
Expand All @@ -93,75 +102,71 @@ klonol -h

## Usage

### Setting the environment variables
### Setting the variables

For GitHub the following variables need to be set.
The following variables need to be set in a file called `credentials.toml`.

| Name | Description | Compulsory |
| ------------------- | ---------------------------------------------------------------------------------------- | ---------- |
| GITHUB_USERNAME | The username whose repositories are to be queried | Yes |
| GITHUB_ACCESS_TOKEN | The personal access token generated previously | Yes |
| GITHUB_BASE_URL | The base domain to be used to test SSH and make API calls from. Defaults to `github.com` | No |
| Name | Description | Compulsory |
|----------------|------------------------------------------------------------------------------------------|------------|
| `username` | The GitHub or Gitea username whose repositories are to be queried | Yes |
| `access_token` | The personal access token generated previously | Yes |
| `base_url` | The base domain to be used to test SSH and make API calls from. Defaults to `github.com` | For Gitea |

For Gitea the following need to be set:
A sample `credentials.toml` file will look like this:

| Name | Description | Compulsory |
| ------------------ | -------------------------------------------------------------------------------------------- | ---------- |
| GITEA_USERNAME | The username whose repositories are to be queried | Yes |
| GITEA_ACCESS_TOKEN | The personal access token generated previously | Yes |
| GITEA_BASE_URL | The domain where the Gitea instance is hosted. Do not include the protocol (e.g. `https://`) | Yes |
hungrybluedev marked this conversation as resolved.
Show resolved Hide resolved
```toml
[github]
username = "your_username"
access_token = "XYZXYZ"

For Unix-like systems, copy the `.env.sample` file, fill in the appropriate
values, comment out the ones you don't need. Then run the following to add
the required variables to your session:
[gitea]
base_url = "git.yourdomain.com"
username = "your_username"
access_token = "XYZXYZ"

```bash
source .env
```

For Windows, you need to set the environment variables manually (for now).
If you want to make it analogous to the Unix way, please contribute a solution.

### Running klonol

Once the environment variables have been set, klonol will retrieve the
relevant information automatically.
Once the variables have been set, klonol will retrieve the relevant
information automatically.

**Help Information**

```bash
# Get the version
klonol --version
# output:
# klonol 0.3.x
# klonol 0.5.x


# Get detailed usage information
klonol -h
# OR
klonol --help
# output:
# klonol 0.3.x
# klonol 0.5.x
# -----------------------------------------------
# Usage: klonol [options] [ARGS]

#
# Description: A CLI tool to "clone all" repositories belonging to you.

#
# klonol requires Access Tokens to work properly. Refer to README for more
# information. It retrieves information about ALL available repositories
# belonging to the authenticated user. Both public and private.

#
# Please follow safety precautions when storing access tokens, and read
# the instructions in README carefully.


#
#
# Options:
# -p, --provider <string> git provider to use
# -a, --action <string> action to perform
# -p, --provider <string> git provider to use (default is github)
# -c, --credentials <string>
# path to credentials.toml file (default is ./credentials.toml)
# -a, --action <string> action to perform [list, clone, pull]
# -v, --verbose enable verbose output
# -h, --help display this help and exit
# --version output version information and exit

```

**Sample usage flow for GitHub**
Expand All @@ -170,6 +175,9 @@ klonol --help
# Navigate to a directory to store all the repositories in
cd ~/Backups/GitHub

# Make sure you have the proper 'credentials.toml' file in the directory
nano credentials.toml

# List all available repositories
klonol

Expand All @@ -189,6 +197,9 @@ klonol -a pull
# Navigate to a directory to store all the repositories in
cd ~/Backups/Gitea

# Make sure you have the proper 'credentials.toml' file in the directory
nano credentials.toml

# List all available repositories
klonol --provider gitea
# OR
Expand Down Expand Up @@ -217,9 +228,26 @@ klonol is already added to PATH.

This project is distributed under the [MIT License](LICENSE).

[workflowbadge]: https://github.com/hungrybluedev/klonol/actions/workflows/ci.yml/badge.svg
[licensebadge]: https://img.shields.io/badge/License-MIT-blue.svg
[workflowurl]: https://github.com/hungrybluedev/klonol/actions/workflows/ci.yml
[licenseurl]: https://github.com/hungrybluedev/klonol/blob/main/LICENSE
[gittagurl]: https://github.com/hungrybluedev/klonol/tags
[gittagbadge]: https://img.shields.io/github/v/tag/hungrybluedev/klonol?color=purple&include_prereleases&sort=semver
[workflow_badge]: https://github.com/hungrybluedev/klonol/actions/workflows/ci.yml/badge.svg

[license_badge]: https://img.shields.io/badge/License-MIT-blue.svg

[workflow_url]: https://github.com/hungrybluedev/klonol/actions/workflows/ci.yml

[license_url]: https://github.com/hungrybluedev/klonol/blob/main/LICENSE

[git_tag_url]: https://github.com/hungrybluedev/klonol/tags

[git_tag_badge]: https://img.shields.io/github/v/tag/hungrybluedev/klonol?color=purple&include_prereleases&sort=semver

## Acknowledgements

- Thanks to [A1ex-N][a1ex_n] for contributing the [idea][toml_idea] and
[initial code][toml_pr] for supporting cross-platform TOML in favour of
unix-specific ENV!

[a1ex_n]: https://github.com/A1ex-N

[toml_idea]: https://github.com/hungrybluedev/klonol/issues/1

[toml_pr]: https://github.com/hungrybluedev/klonol/pull/2
18 changes: 16 additions & 2 deletions build.vsh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import os

println('Removing old artifacts...')
rmdir_all('bin') or {}
println('Done removing "bin" directory.')
Expand All @@ -7,9 +9,21 @@ mkdir('bin')!
println('Done creating "bin" directory.')

println('\nChecking if everything is formatted correctly...')
execute_or_panic('v fmt -verify .')
execute_or_panic('${quoted_path(@VEXE)} fmt -verify .')
println('Done checking formatting.')

println('\nCompiling and building executable...')
execute_or_panic('v -prod . -o bin/klonol')

if '-fast' in os.args {
execute_or_panic('${quoted_path(@VEXE)} . -o bin/klonol')
} else {
// Make compilation on windows faster
compiler := $if windows {
'msvc'
} $else {
'cc'
}
execute_or_panic('${quoted_path(@VEXE)} -cc "${compiler}" -prod . -o bin/klonol')
}

println('Done compiling and placing executable in "bin".')
35 changes: 32 additions & 3 deletions common/types.v
Original file line number Diff line number Diff line change
@@ -1,12 +1,41 @@
module common

import x.json2
import strings

pub struct Credentials {
pub enum Provider {
github
gitea
}

pub struct CredentialFile {
pub:
credentials []Credential
}

pub fn (file CredentialFile) to_toml() string {
mut builder := strings.new_builder(128)
for cred in file.credentials {
builder.write_string(cred.to_toml())
}
return builder.str()
}

pub struct Credential {
pub:
provider Provider [required]
base_url string = 'github.com'
username string
access_token string = 'unset value'
username string [required]
access_token string [required]
}

fn (c Credential) to_toml() string {
return '
[${c.provider.str()}]
base_url: ${c.base_url}
username: ${c.username}
access_token: ${c.access_token}
'
}

pub struct Repository {
Expand Down
Loading