Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add GitLab Release badge, run all [GitLab] #7021

Merged
merged 8 commits into from
Oct 23, 2021
46 changes: 46 additions & 0 deletions services/gitlab/gitlab-base.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,50 @@ export default class GitLabBase extends BaseJsonService {
})
)
}

async fetchPage({ page, requestParams, schema }) {
const { res, buffer } = await this._request({
...requestParams,
...{ options: { qs: { page } } },
})

const json = this._parseJson(buffer)
const data = this.constructor._validate(json, schema)
return { res, data }
}

async fetchPaginatedArrayData({
url,
options,
schema,
errorMessages,
firstPageOnly = false,
}) {
const requestParams = this.authHelper.withBasicAuth({
url,
options: {
headers: { Accept: 'application/json' },
qs: { per_page: 100 },
...options,
},
errorMessages,
})

const {
res: { headers },
data,
} = await this.fetchPage({ page: 1, requestParams, schema })
const numberOfPages = headers['x-total-pages']

if (numberOfPages === 1 || firstPageOnly) {
return data
}

const pageData = await Promise.all(
[...Array(numberOfPages - 1).keys()].map((_, i) =>
this.fetchPage({ page: ++i + 1, requestParams, schema })
)
)
return [...data].concat(...pageData)
}
Comment on lines +31 to +64
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitLab's max per page value is 100, and I saw that exceeded in several cases just with the releases on the handful of projects I tested with. A cursory glance through the GitLab API docs makes it seem a very common response structure will be pages of arrays with the page keys in the response headers.

Opted to go ahead and pull that into the base class in anticipation of other APIs. Added it with relevant short circuiting logic (since GitLab doesn't seem to have a separate "latest" API endpoint that returns a single object like GitHub does in some cases), and also requesting all the pages in parallel since it's both possible to do so and preferable to 1 by 1

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👍

I know this is a bit of a fiddly thing to write a test for, but if we're going to be relying on this logic in a number of place it would be good to have it under test

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The service tests include a target (the GitLab repo itself) which has more than 100 releases so it is observable (albeit manually via TRACE_SERVICES). Probably some class level unit testing is possible with sufficient mocks, but feel like that's probably best investigated/implemented in a separate PR, thoughts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a plan 👍

}
152 changes: 152 additions & 0 deletions services/gitlab/gitlab-release.service.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
import Joi from 'joi'
import { optionalUrl } from '../validators.js'
import { latest, renderVersionBadge } from '../version.js'
import { NotFound } from '../index.js'
import GitLabBase from './gitlab-base.js'

const schema = Joi.array().items(
Joi.object({
name: Joi.string().required(),
tag_name: Joi.string().required(),
})
)

const queryParamSchema = Joi.object({
gitlab_url: optionalUrl,
include_prereleases: Joi.equal(''),
sort: Joi.string().valid('date', 'semver').default('date'),
display_name: Joi.string().valid('tag', 'release').default('tag'),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Figured it made sense to add this from day 0 given where we ended up with this on GitHub. I've opted to set the default to tag to be consistent with what GitHub currently is but don't have strong opinions on this one way or another.

date_order_by: Joi.string()
.valid('created_at', 'released_at')
.default('created_at'),
}).required()

const documentation = `
<p>
You may use your GitLab Project Id (e.g. 25813592) or your Project Path (e.g. megabyte-labs/dockerfile/ci-pipeline/ansible-lint)
</p>
`
const commonProps = {
namedParams: {
project: 'shields-ops-group/tag-test',
},
documentation,
}

export default class GitLabRelease extends GitLabBase {
static category = 'version'

static route = {
base: 'gitlab/v/release',
pattern: ':project+',
queryParamSchema,
}

static examples = [
{
title: 'GitLab Release (latest by date)',
...commonProps,
queryParams: { sort: 'date', date_order_by: 'created_at' },
staticPreview: renderVersionBadge({ version: 'v2.0.0' }),
},
{
title: 'GitLab Release (latest by SemVer)',
...commonProps,
queryParams: { sort: 'semver' },
staticPreview: renderVersionBadge({ version: 'v4.0.0' }),
},
{
title: 'GitLab Release (latest by SemVer pre-release)',
...commonProps,
queryParams: {
sort: 'semver',
include_prereleases: null,
},
staticPreview: renderVersionBadge({ version: 'v5.0.0-beta.1' }),
},
{
title: 'GitLab Release (custom instance)',
namedParams: {
project: 'GNOME/librsvg',
},
documentation,
queryParams: {
sort: 'semver',
include_prereleases: null,
gitlab_url: 'https://gitlab.gnome.org',
date_order_by: 'created_at',
},
staticPreview: renderVersionBadge({ version: 'v2.51.4' }),
},
{
title: 'GitLab Release (by release name)',
namedParams: {
project: 'gitlab-org/gitlab',
},
documentation,
queryParams: {
sort: 'semver',
include_prereleases: null,
gitlab_url: 'https://gitlab.com',
display_name: 'release',
date_order_by: 'created_at',
},
staticPreview: renderVersionBadge({ version: 'GitLab 14.2' }),
},
]

static defaultBadgeData = { label: 'release' }

async fetch({ project, baseUrl, isSemver, orderBy }) {
// https://docs.gitlab.com/ee/api/releases/
return this.fetchPaginatedArrayData({
schema,
url: `${baseUrl}/api/v4/projects/${encodeURIComponent(project)}/releases`,
errorMessages: {
404: 'project not found',
},
options: {
qs: { order_by: orderBy },
},
firstPageOnly: !isSemver,
})
}

static transform({ releases, isSemver, includePrereleases, displayName }) {
if (releases.length === 0) {
throw new NotFound({ prettyMessage: 'no releases found' })
}

const displayKey = displayName === 'tag' ? 'tag_name' : 'name'

if (!isSemver) {
return releases[0][displayKey]
}

return latest(
releases.map(t => t[displayKey]),
{ pre: includePrereleases }
)
}

async handle(
{ project },
{
gitlab_url: baseUrl = 'https://gitlab.com',
include_prereleases: pre,
sort,
display_name: displayName,
date_order_by: orderBy,
}
) {
const isSemver = sort === 'semver'
const releases = await this.fetch({ project, baseUrl, isSemver, orderBy })
const version = this.constructor.transform({
releases,
isSemver,
includePrereleases: pre !== undefined,
displayName,
})
return renderVersionBadge({ version })
}
}
49 changes: 49 additions & 0 deletions services/gitlab/gitlab-release.tester.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
import { isSemver, withRegex } from '../test-validators.js'
import { createServiceTester } from '../tester.js'
export const t = await createServiceTester()

const isGitLabDisplayVersion = withRegex(/^GitLab [1-9][0-9]*.[0-9]*/)

t.create('Release (latest by date)')
.get('/shields-ops-group/tag-test.json')
.expectBadge({ label: 'release', message: 'v2.0.0', color: 'blue' })

t.create('Release (nested groups latest by date)')
.get('/gitlab-org/frontend/eslint-plugin.json')
.expectBadge({ label: 'release', message: isSemver, color: 'blue' })

t.create('Release (latest by date, order by created_at)')
.get('/shields-ops-group/tag-test.json?date_order_by=created_at')
.expectBadge({ label: 'release', message: 'v2.0.0', color: 'blue' })

t.create('Release (latest by date, order by released_at)')
.get('/shields-ops-group/tag-test.json?date_order_by=released_at')
.expectBadge({ label: 'release', message: 'v2.0.0', color: 'blue' })

t.create('Release (project id latest by date)')
.get('/29538796.json')
.expectBadge({ label: 'release', message: 'v2.0.0', color: 'blue' })

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an example of a gitlab project which is nested in a group and has releases we could use for a service test: https://gitlab.com/gitlab-org/frontend/eslint-plugin/-/releases

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Excellent find and/or memory 😆

t.create('Release (latest by semver)')
.get('/shields-ops-group/tag-test.json?sort=semver')
.expectBadge({ label: 'release', message: 'v4.0.0', color: 'blue' })

t.create('Release (latest by semver pre-release)')
.get('/shields-ops-group/tag-test.json?sort=semver&include_prereleases')
.expectBadge({ label: 'release', message: 'v5.0.0-beta.1', color: 'orange' })

t.create('Release (release display name)')
.get('/gitlab-org/gitlab.json?display_name=release')
.expectBadge({ label: 'release', message: isGitLabDisplayVersion })

t.create('Release (custom instance')
.get('/GNOME/librsvg.json?gitlab_url=https://gitlab.gnome.org')
.expectBadge({ label: 'release', message: isSemver, color: 'blue' })

t.create('Release (project not found)')
.get('/fdroid/nonexistant.json')
.expectBadge({ label: 'release', message: 'project not found' })

t.create('Release (no tags)')
.get('/fdroid/fdroiddata.json')
.expectBadge({ label: 'release', message: 'no releases found' })