Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option output_action #178

Merged
merged 38 commits into from
Mar 8, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
2b014c6
Update index.mjs
lowlighter Mar 7, 2021
f36b2b6
Update index.mjs
lowlighter Mar 7, 2021
da9748e
Update index.mjs
lowlighter Mar 7, 2021
82b3a8d
Update index.mjs
lowlighter Mar 7, 2021
5623e77
Update index.mjs
lowlighter Mar 7, 2021
73bee2a
Update index.mjs
lowlighter Mar 8, 2021
f7ce121
Update index.mjs
lowlighter Mar 8, 2021
0c1d7a9
Update
lowlighter Mar 8, 2021
472b40a
Update
lowlighter Mar 8, 2021
166f909
Update index.mjs
lowlighter Mar 8, 2021
b7f1a9f
Update index.mjs
lowlighter Mar 8, 2021
16b4dec
Update index.mjs
lowlighter Mar 8, 2021
fafd48e
Update index.mjs
lowlighter Mar 8, 2021
91c18ca
Update index.mjs
lowlighter Mar 8, 2021
0be8a05
Update index.mjs
lowlighter Mar 8, 2021
0660145
Update index.mjs
lowlighter Mar 8, 2021
4d915d6
Continue
lowlighter Mar 8, 2021
3e230b0
Update index.mjs
lowlighter Mar 8, 2021
751378e
Update index.mjs
lowlighter Mar 8, 2021
c327324
Update index.mjs
lowlighter Mar 8, 2021
a3e910f
Update
lowlighter Mar 8, 2021
33302fc
Update index.mjs
lowlighter Mar 8, 2021
c4e6fa1
Update action
lowlighter Mar 8, 2021
125b57c
Update action
lowlighter Mar 8, 2021
7c21076
Update index.mjs
lowlighter Mar 8, 2021
8eae216
Update index.mjs
lowlighter Mar 8, 2021
50b3316
Update index.mjs
lowlighter Mar 8, 2021
69f1090
Update index.mjs
lowlighter Mar 8, 2021
1f2c601
Update
lowlighter Mar 8, 2021
438014c
Debug
lowlighter Mar 8, 2021
589e584
Remove debug
lowlighter Mar 8, 2021
0f5e384
Update index.mjs
lowlighter Mar 8, 2021
c34c4e5
Update index.mjs
lowlighter Mar 8, 2021
58e9a0b
Update index.mjs
lowlighter Mar 8, 2021
971da69
Update index.mjs
lowlighter Mar 8, 2021
131d4ec
Update index.mjs
lowlighter Mar 8, 2021
4a94d8f
Some comments changes
lowlighter Mar 8, 2021
5a80d87
Remove action.yml
lowlighter Mar 8, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 15 additions & 16 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -182,12 +182,11 @@ inputs:
description: Use mocked data instead of live APIs
default: no

# Use a pre-built image from GitHub registry when using unreleased versions of "lowlighter/metrics"
# This option has no effect on forks (images will always be rebuilt from Dockerfile)
# Use a pre-built image from GitHub registry (experimental)
# See https://github.com/users/lowlighter/packages/container/package/metrics for more information
use_prebuilt_image:
description: Use pre-built image from GitHub registry
default: yes
default: ""

# ====================================================================================
# 📰 Recent activity
Expand Down Expand Up @@ -847,7 +846,6 @@ runs:
steps:
- run: |
# Create environment file from inputs and GitHub variables
echo "::group::Metrics docker image setup"
cd $METRICS_ACTION_PATH
touch .env
for INPUT in $(echo $INPUTS | jq -r 'to_entries|map("INPUT_\(.key|ascii_upcase)=\(.value|@uri)")|.[]'); do
Expand All @@ -866,11 +864,20 @@ runs:

# Image tag (extracted from version or from env)
METRICS_TAG=v$(echo $METRICS_VERSION | sed -r 's/^([0-9]+[.][0-9]+).*/\1/')
if [[ $METRICS_USE_PREBUILT_IMAGE ]]; then
METRICS_TAG=$METRICS_USE_PREBUILT_IMAGE
echo "Pre-built image: yes"
fi
echo "Image tag: $METRICS_TAG"

# Image name
# Pre-built image
if [[ $METRICS_USE_PREBUILT_IMAGE ]]; then
echo "Using pre-built version $METRICS_TAG, will pull docker image from GitHub registry"
METRICS_IMAGE=ghcr.io/lowlighter/metrics:$METRICS_TAG
docker image pull $METRICS_IMAGE > /dev/null
# Official action
if [[ $METRICS_SOURCE == "lowlighter" ]]; then
elif [[ $METRICS_SOURCE == "lowlighter" ]]; then
# Is released version
set +e
METRICS_IS_RELEASED=$(expr $(expr match $METRICS_VERSION .*-beta) == 0)
Expand All @@ -880,14 +887,7 @@ runs:
if [[ "$METRICS_IS_RELEASED" -gt "0" ]]; then
echo "Using released version $METRICS_TAG, will pull docker image from GitHub registry"
METRICS_IMAGE=ghcr.io/lowlighter/metrics:$METRICS_TAG
docker image pull $METRICS_IMAGE
# Use registry for unreleased version with pre-built images
elif [[ ! $METRICS_USE_PREBUILT_IMAGE =~ ^([Ff]alse|[Oo]ff|[Nn]o|0)$ ]]; then
METRICS_TAG="$METRICS_TAG-beta"
echo "Image tag (updated): $METRICS_TAG"
echo "Using pre-built version $METRICS_TAG, will pull docker image from GitHub registry"
METRICS_IMAGE=ghcr.io/lowlighter/metrics:$METRICS_TAG
docker image pull $METRICS_IMAGE
docker image pull $METRICS_IMAGE > /dev/null
# Rebuild image for unreleased version
else
echo "Using an unreleased version ($METRICS_VERSION)"
Expand All @@ -902,16 +902,15 @@ runs:

# Build image if necessary
set +e
docker image inspect $METRICS_IMAGE
docker image inspect $METRICS_IMAGE > /dev/null
METRICS_IMAGE_NEEDS_BUILD="$?"
set -e
if [[ "$METRICS_IMAGE_NEEDS_BUILD" -gt "0" ]]; then
echo "Image $METRICS_IMAGE is not present locally, rebuilding it from Dockerfile"
docker build -t $METRICS_IMAGE .
docker build -t $METRICS_IMAGE . > /dev/null
else
echo "Image $METRICS_IMAGE is present locally"
fi
echo "::endgroup::"

# Run docker image with current environment
docker run --init --volume $GITHUB_EVENT_PATH:$GITHUB_EVENT_PATH --env-file .env $METRICS_IMAGE
Expand Down
9 changes: 7 additions & 2 deletions source/app/action/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,12 @@ runs:
echo $INPUT >> .env
done
env | grep -E '^(GITHUB|ACTIONS|CI)' >> .env
echo "Environment variable: loaded"
echo "Environment variables: loaded"

# Renders output folder
METRICS_RENDERS="/metrics_renders"
sudo mkdir -p $METRICS_RENDERS
echo "Renders output folder: $METRICS_RENDERS"

# Source repository (picked from action name)
METRICS_SOURCE=$(echo $METRICS_ACTION | sed -E 's/metrics.*?$//g')
Expand Down Expand Up @@ -94,7 +99,7 @@ runs:
echo "::endgroup::"

# Run docker image with current environment
docker run --init --volume $GITHUB_EVENT_PATH:$GITHUB_EVENT_PATH --env-file .env $METRICS_IMAGE
docker run --init --volume $GITHUB_EVENT_PATH:$GITHUB_EVENT_PATH --volume $METRICS_RENDERS:/renders --env-file .env $METRICS_IMAGE
rm .env
shell: bash
env:
Expand Down
76 changes: 71 additions & 5 deletions source/app/action/index.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
import setup from "../metrics/setup.mjs"
import mocks from "../mocks/index.mjs"
import metrics from "../metrics/index.mjs"
import fs from "fs/promises"
import paths from "path"
process.on("unhandledRejection", error => { throw error }) //eslint-disable-line max-statements-per-line, brace-style

//Debug message buffer
Expand Down Expand Up @@ -40,6 +42,10 @@
console.log("Skipped because [Skip GitHub Action] is in commit message")
process.exit(0)
}
if (/Auto-generated metrics for run #\d+/.test(github.context.payload.head_commit.message)) {
console.log("Skipped because this seems to be an automated pull request merge")
process.exit(0)
}
}

//Load configuration
Expand All @@ -58,6 +64,7 @@
"committer.token":_token, "committer.branch":_branch,
"use.prebuilt.image":_image,
retries, "retries.delay":retries_delay,
"output.action":_action,
...config
} = metadata.plugins.core.inputs.action({core})
const q = {...query, ...(_repo ? {repo:_repo} : null), template}
Expand All @@ -73,6 +80,7 @@
DEBUG = false
}
info("Debug flags", dflags)
q["debug.flags"] = dflags.join(" ")

//Token for data gathering
info("GitHub token", token, {token:true})
Expand Down Expand Up @@ -112,13 +120,17 @@
const committer = {}
if (!dryrun) {
//Compute committer informations
committer.commit = true
committer.token = _token || token
committer.commit = true
committer.pr = /^pull-request/.test(_action)
committer.merge = _action.match(/^pull-request-(?<method>merge|squash|rebase)$/)?.groups?.method ?? null
committer.branch = _branch || github.context.ref.replace(/^refs[/]heads[/]/, "")
committer.head = committer.pr ? `metrics-run-${github.context.runId}` : committer.branch
info("Committer token", committer.token, {token:true})
if (!committer.token)
throw new Error("You must provide a valid GitHub token to commit your metrics")
info("Committer branch", committer.branch)
info("Committer head branch", committer.head)
//Instantiate API for committer
committer.rest = github.getOctokit(committer.token)
info("Committer REST API", "ok")
Expand All @@ -128,13 +140,29 @@
catch {
info("Committer account", "(github-actions)")
}
//Create head branch if needed
try {
await committer.rest.git.getRef({...github.context.repo, ref:`heads/${committer.head}`})
info("Committer head branch status", "ok")
}
catch (error) {
console.debug(error)
if (/not found/i.test(`${error}`)) {
const {data:{object:{sha}}} = await committer.rest.git.getRef({...github.context.repo, ref:`heads/${committer.branch}`})
info("Committer branch current sha", sha)
await committer.rest.git.createRef({...github.context.repo, ref:`refs/heads/${committer.head}`, sha})
info("Committer head branch status", "(created)")
}
else
throw error
}
//Retrieve previous render SHA to be able to update file content through API
committer.sha = null
try {
const {repository:{object:{oid}}} = await graphql(`
query Sha {
repository(owner: "${github.context.repo.owner}", name: "${github.context.repo.repo}") {
object(expression: "${committer.branch}:${filename}") { ... on Blob { oid } }
object(expression: "${committer.head}:${filename}") { ... on Blob { oid } }
}
}
`, {headers:{authorization:`token ${committer.token}`}})
Expand Down Expand Up @@ -204,7 +232,7 @@
info.break()
info.section("Rendering")
let error = null, rendered = null
for (let attempt = 0; attempt < retries; attempt++) {
for (let attempt = 1; attempt <= retries; attempt++) {
try {
console.debug(`::group::Attempt ${attempt}/${retries}`)
;({rendered} = await metrics({login:user, q}, {graphql, rest, plugins, conf, die, verify, convert}, {Plugins, Templates}))
Expand All @@ -222,15 +250,53 @@
throw error ?? new Error("Could not render metrics")
info("Status", "complete")

//Save output to renders output folder
info.break()
info.section("Saving")
await fs.writeFile(paths.join("/renders", filename), Buffer.from(rendered))
info(`Save to /metrics_renders/${filename}`, "ok")

//Commit metrics
if (committer.commit) {
await committer.rest.repos.createOrUpdateFileContents({
...github.context.repo, path:filename, message:`Update ${filename} - [Skip GitHub Action]`,
content:Buffer.from(rendered).toString("base64"),
branch:committer.branch,
branch:committer.pr ? committer.head : committer.branch,
...(committer.sha ? {sha:committer.sha} : {}),
})
info("Commit to repository", "success")
info(`Commit to branch ${committer.branch}`, "ok")
}

//Pull request
if (committer.pr) {
//Create pull request
let number = null
try {
({data:{number}} = await committer.rest.pulls.create({...github.context.repo, head:committer.head, base:committer.branch, title:`Auto-generated metrics for run #${github.context.runId}`, body:" ", maintainer_can_modify:true}))
info(`Pull request from ${committer.head} to ${committer.branch}`, "(created)")
}
catch (error) {
console.debug(error)
if (/A pull request already exists/.test(error)) {
info(`Pull request from ${committer.head} to ${committer.branch}`, "(already existing)")
const q = `repo:${github.context.repo.owner}/${github.context.repo.repo}+type:pr+state:open+Auto-generated metrics for run #${github.context.runId}+in:title`
const prs = (await committer.rest.search.issuesAndPullRequests({q})).data.items.filter(({user:{login}}) => login === "github-actions[bot]")
if (prs.length < 1)
throw new Error("0 matching prs. Cannot preoceed.")
if (prs.length > 1)
throw new Error(`Found more than one matching prs: ${prs.map(({number}) => `#${number}`).join(", ")}. Cannot proceed.`)
;({number} = prs.shift())
}
else
throw error
}
info("Pull request number", number)
//Merge pull request
if (committer.merge) {
info("Merge method", committer.merge)
await committer.rest.pulls.merge({...github.context.repo, pull_number:number, merge_method:committer.merge})
info(`Merge #${number} to ${committer.branch}`, "ok")
}
}

//Success
Expand Down
32 changes: 32 additions & 0 deletions source/plugins/core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,38 @@ Specify a single value to apply it to both height and with, and two values to us
config_padding: 6%, 10% # 6% width padding, 10% height padding
```

### 🧶 Using commits, pull requests or manual review to handle metrics output

It is possible to configure output behaviour using `output_action` option, which can be set to:
- `none`, where output will be generated in `/rendered/${filename}` without being pushed
- You can then manually post-process it
- `commit` (default), where output will directly be committed and pushed to `committer_branch`
- `pull-request`, where output will be committed to a new branch with current run id waiting for to be merged in `committer_branch`
- By appending either `-merge`, `-squash` or `-rebase`, pull request will be automatically merged with given method
- This method is useful to combine all editions of a single run with multiples metrics steps into a single commit on targetted branch
- If you choose to manually merge pull requests, be sure to disable `push:` triggers on your workflow, as it'll count as your own commit

#### ℹ️ Examples workflows

```yaml
# The following will:
# - open a pull request with "my-metrics-0.svg" as first commit
# - append "my-metrics-1.svg" as second commit
# - merge pull request (as second step is set to "pull-request-merge")

- uses: lowlighter/metrics@latest
with:
# ... other options
filename: my-metrics-0.svg
output_action: pull-request

- uses: lowlighter/metrics@latest
with:
# ... other options
filename: my-metrics-1.svg
output_action: pull-request-merge
```

### ♻️ Retrying automatically failed rendering

Rendering is subject to external factors and can fail from time to time.
Expand Down
26 changes: 20 additions & 6 deletions source/plugins/core/metadata.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,19 @@ inputs:
type: string
default: github-metrics.svg

# Output action
output_action:
description: Output action
type: string
default: commit
values:
- none # Only generate file in "/metrics_renders"
- commit # Commit output to "committer_branch"
- pull-request # Commit output to a new branch and open a pull request to "committer_branch"
- pull-request-merge # Same as "pull-request" and additionaly merge pull request
- pull-request-squash # Same as "pull-request" and additionaly squash and merge pull request
- pull-request-rebase # Same as "pull-request" and additionaly rebase and merge pull request

# Optimize SVG image to reduce its filesize
# Some templates may not support this option
optimize:
Expand Down Expand Up @@ -148,11 +161,11 @@ inputs:

# Time to wait (in seconds) before each retry
retries_delay:
description: Time to wait (in seconds) before each retry
type: number
default: 300
min: 0
max: 3600
description: Time to wait (in seconds) before each retry
type: number
default: 300
min: 0
max: 3600

# ====================================================================================
# Options below are mostly used for testing
Expand Down Expand Up @@ -189,7 +202,8 @@ inputs:
- --halloween
- --error

# Dry-run mode (perform generation without pushing it)
# Dry-run mode (perform generation without output)
# Unlike "output_action" set to "none", output file won't be available in "/metrics_renders"
dryrun:
description: Enable dry-run
type: boolean
Expand Down