Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gomod): Notify extra packages updated by "go get" #28938

Merged
merged 63 commits into from
May 27, 2024
Merged
Show file tree
Hide file tree
Changes from 58 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
2f5cb33
feat(gomod): Notify extra packages updated by "go get"
zharinov May 8, 2024
0564d7c
feat: Support custom artifact notice
zharinov May 9, 2024
0109e98
Update lib/workers/repository/update/branch/get-updated.spec.ts
zharinov May 10, 2024
be2425e
Merge branch 'main' into feat/artifact-notice-comments
zharinov May 10, 2024
dfd0f2d
Update lib/workers/repository/update/branch/index.ts
zharinov May 10, 2024
8c24ae8
Merge branch 'main' into feat/artifact-notice-comments
zharinov May 10, 2024
24743e5
Fixes
zharinov May 10, 2024
7a895ee
Update lib/workers/repository/update/branch/index.ts
rarkins May 13, 2024
12d01a4
Merge branch 'feat/artifact-notice-comments' into feat/gomod-notify-e…
zharinov May 13, 2024
ed2d0b3
Fix
zharinov May 13, 2024
0a70ec0
Merge branch 'main' into feat/artifact-notice-comments
zharinov May 13, 2024
703294a
Fix
zharinov May 13, 2024
5ff415f
Fix prettier
zharinov May 13, 2024
7d83eb7
Fix
zharinov May 13, 2024
c86fb59
Merge branch 'main' into feat/gomod-notify-extra-packages
zharinov May 13, 2024
7aa1c51
Fix dependency
zharinov May 13, 2024
c7c8fab
Merge branch 'feat/artifact-notice-comments' into feat/gomod-notify-e…
zharinov May 13, 2024
684a58c
Merge branch 'main' into feat/gomod-notify-extra-packages
zharinov May 13, 2024
3a0a770
feat(datasource/kubernetes-api): add flux versions from flux 2.3.0 (#…
QuentinBtd May 13, 2024
9c11ad9
chore(deps): update dependency @swc/core to v1.5.5 (#29040)
renovate[bot] May 13, 2024
ff91408
feat(pip_requirements): Extract flags from package files with no deps…
mbudnek May 13, 2024
f65e602
chore(deps): update dependency @types/node to v18.19.32 (#29050)
renovate[bot] May 14, 2024
27f0237
build(deps): update dependency validate-npm-package-name to v5.0.1 (#…
renovate[bot] May 14, 2024
7533fd4
chore(deps): update dependency @types/diff to v5.2.1 (#29046)
renovate[bot] May 14, 2024
64aef03
build(deps): update dependency re2 to v1.20.11 (#29053)
renovate[bot] May 14, 2024
f4bf0c5
feat(corepack): Add env COREPACK_INTEGRITY_KEYS (#29047)
Keysox May 14, 2024
98443cf
docs: add modules introduction (#29038)
viceice May 14, 2024
2487e27
feat(gitlab): retry requests on resource locks (#29019)
fgreinacher May 14, 2024
e44c084
feat(util/yaml): replace more go templates (#29061)
viceice May 14, 2024
585612c
chore(deps): update dependency @types/node to v18.19.33 (#29051)
renovate[bot] May 14, 2024
0126a2c
feat(datasource/kubernetes-api): add missing image api kinds (#29062)
QuentinBtd May 14, 2024
795d254
fix(azure): paginate getTeams() (#29060)
ddovile May 14, 2024
e1475d7
feat: allow templating of the extends field (#27955)
lstoeferle May 14, 2024
1029e2f
fix(datasource/docker): use digest from header if available (#29065)
viceice May 14, 2024
2ffa7d2
build(deps): update dependency semver to v7.6.1 (#29073)
renovate[bot] May 14, 2024
a4422f8
build(deps): update opentelemetry-js monorepo (#29074)
renovate[bot] May 14, 2024
9ec7392
build(deps): update dependency semver to v7.6.2 (#29076)
renovate[bot] May 14, 2024
45cdf08
chore(deps): update codecov/codecov-action action to v4.4.0 (#29080)
renovate[bot] May 15, 2024
bedce3d
fix(deps): update ghcr.io/renovatebot/base-image docker tag to v2.11.…
renovate[bot] May 15, 2024
ae1c490
build(deps): update dependency zod to v3.23.7 (#29077)
renovate[bot] May 15, 2024
c3fef27
fix(datasource/github-runners): add Ubuntu 24.04 Noble Numbat as unst…
HonkingGoose May 15, 2024
6de5746
chore(deps): update ghcr.io/containerbase/devcontainer docker tag to …
renovate[bot] May 15, 2024
2ca22c8
build(deps): update dependency zod to v3.23.8 (#29090)
renovate[bot] May 15, 2024
be27e72
docs: Update Swissquote article with information on the scheduler and…
onigoetz May 15, 2024
270df5b
docs(bot comparison): dependabot-core switched to MIT license (#29095)
HonkingGoose May 15, 2024
50e3790
fix(deps): update ghcr.io/containerbase/sidecar docker tag to v10.6.1…
renovate[bot] May 15, 2024
9a84eaf
docs(config): add note about GnuPG v2.4 usage (#29067)
viceice May 15, 2024
8680ef3
fix(deps): update ghcr.io/renovatebot/base-image docker tag to v2.11.…
renovate[bot] May 15, 2024
3a5e8c4
Merge branch 'main' into feat/gomod-notify-extra-packages
zharinov May 15, 2024
441729d
Merge branch 'main' into feat/gomod-notify-extra-packages
zharinov May 16, 2024
1d5757f
Revert irrelevant space change
zharinov May 16, 2024
897d76f
Refactor
zharinov May 16, 2024
19ff14f
Add tests
zharinov May 16, 2024
19771e2
Use "markdown-table"
zharinov May 16, 2024
e88ed91
Small test fix
zharinov May 16, 2024
70f16c1
Add one more test
zharinov May 16, 2024
39c04df
Merge branch 'main' into feat/gomod-notify-extra-packages
zharinov May 16, 2024
77086d3
Merge branch 'main' into feat/gomod-notify-extra-packages
zharinov May 18, 2024
bcf0d68
Change message format
zharinov May 18, 2024
0af0f71
Fix test
zharinov May 18, 2024
6468cf7
Merge branch 'main' into feat/gomod-notify-extra-packages
zharinov May 22, 2024
de43be4
Fix coverage
zharinov May 19, 2024
d77412f
Merge branch 'main' into feat/gomod-notify-extra-packages
zharinov May 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions lib/modules/manager/gomod/artifacts-extra.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
import { codeBlock } from 'common-tags';
import {
extraDepsTable,
getExtraDeps,
getExtraDepsNotice,
} from './artifacts-extra';
import type { ExtraDep } from './types';

describe('modules/manager/gomod/artifacts-extra', () => {
const goModBefore = codeBlock`
go 1.22.2

require (
github.com/foo/foo v1.0.0
github.com/bar/bar v2.0.0
)

replace baz/baz => qux/qux
`;

const goModAfter = codeBlock`
go 1.22.2

// Note the order change
require (
github.com/bar/bar v2.2.2
github.com/foo/foo v1.1.1
)

replace baz/baz => quux/quux
`;

describe('getExtraDeps', () => {
it('detects extra dependencies', () => {
const excludeDeps = ['github.com/foo/foo'];

const res = getExtraDeps(goModBefore, goModAfter, excludeDeps);

expect(res).toEqual([
{
depName: 'github.com/bar/bar',
currentValue: 'v2.0.0',
newValue: 'v2.2.2',
},
] satisfies ExtraDep[]);
});
});

describe('extraDepsTable', () => {
it('generates a table', () => {
const extraDeps: ExtraDep[] = [
{
depName: 'github.com/foo/foo',
currentValue: 'v1.0.0',
newValue: 'v1.1.1',
},
{
depName: 'github.com/bar/bar',
currentValue: 'v2.0.0',
newValue: 'v2.2.2',
},
];

const res = extraDepsTable(extraDeps);

expect(res).toEqual(
[
'| **Package** | **Change** |',
'| :------------------- | :------------------- |',
'| `github.com/foo/foo` | `v1.0.0` -> `v1.1.1` |',
'| `github.com/bar/bar` | `v2.0.0` -> `v2.2.2` |',
].join('\n'),
);
});
});

describe('getExtraDepsNotice', () => {
it('returns null when one of files is missing', () => {
expect(getExtraDepsNotice(null, goModAfter, [])).toBeNull();
expect(getExtraDepsNotice(goModBefore, null, [])).toBeNull();
});

it('returns null when all dependencies are excluded', () => {
const excludeDeps = ['github.com/foo/foo', 'github.com/bar/bar'];
const res = getExtraDepsNotice(goModBefore, goModAfter, excludeDeps);
expect(res).toBeNull();
});

it('returns a notice when there are extra dependencies', () => {
const excludeDeps = ['github.com/foo/foo'];

const res = getExtraDepsNotice(goModBefore, goModAfter, excludeDeps);

expect(res).toEqual(
[
'In addition to the dependencies listed above, the following packages will also be updated:',
'',
'',
'| **Package** | **Change** |',
'| :------------------- | :------------------- |',
'| `github.com/bar/bar` | `v2.0.0` -> `v2.2.2` |',
'',
'',
].join('\n'),
);
});
});
});
99 changes: 99 additions & 0 deletions lib/modules/manager/gomod/artifacts-extra.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
import { diffLines } from 'diff';
import markdownTable from 'markdown-table';
import { parseLine } from './line-parser';
import type { ExtraDep } from './types';

export function getExtraDeps(
goModBefore: string,
goModAfter: string,
excludeDeps: string[],
): ExtraDep[] {
const result: ExtraDep[] = [];

const diff = diffLines(goModBefore, goModAfter, {
newlineIsToken: true,
});

const addDeps: Record<string, string> = {};
const rmDeps: Record<string, string> = {};
for (const { added, removed, value } of diff) {
if (!added && !removed) {
continue;
}

const res = parseLine(value);
if (!res) {
continue;
}

const { depName, currentValue } = res;
if (!depName || !currentValue) {
continue;
}

if (added) {
addDeps[depName] = currentValue;
} else {
rmDeps[depName] = currentValue;
}
}

for (const [depName, currentValue] of Object.entries(rmDeps)) {
if (excludeDeps.includes(depName)) {
continue;
}

const newValue = addDeps[depName];
if (newValue) {
result.push({
depName,
currentValue,
newValue,
});
}
}

return result;
}

export function extraDepsTable(extraDeps: ExtraDep[]): string {
const tableLines: string[][] = [];

tableLines.push(['**Package**', '**Change**']);

for (const { depName, currentValue, newValue } of extraDeps) {
const depNameQuoted = `\`${depName}\``;
const versionChangeQuoted = `\`${currentValue}\` -> \`${newValue}\``;
tableLines.push([depNameQuoted, versionChangeQuoted]);
}

return markdownTable(tableLines, {
align: ['l', 'l'],
});
}

export function getExtraDepsNotice(
goModBefore: string | null,
goModAfter: string | null,
excludeDeps: string[],
): string | null {
if (!goModBefore || !goModAfter) {
return null;
}

const extraDeps = getExtraDeps(goModBefore, goModAfter, excludeDeps);
if (extraDeps.length === 0) {
return null;
}

const noticeLines: string[] = [];

noticeLines.push(
'In addition to the dependencies listed above, the following packages will also be updated:',
);
noticeLines.push('\n');
noticeLines.push(extraDepsTable(extraDeps));
noticeLines.push('\n');

return noticeLines.join('\n');
}
44 changes: 44 additions & 0 deletions lib/modules/manager/gomod/artifacts.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import type { StatusResult } from '../../../util/git/types';
import * as _hostRules from '../../../util/host-rules';
import * as _datasource from '../../datasource';
import type { UpdateArtifactsConfig } from '../types';
import * as _artifactsExtra from './artifacts-extra';
import * as gomod from '.';

type FS = typeof import('../../../util/fs');
Expand All @@ -28,11 +29,13 @@ jest.mock('../../../util/fs', () => {
};
});
jest.mock('../../datasource', () => mockDeep());
jest.mock('./artifacts-extra', () => mockDeep());

process.env.CONTAINERBASE = 'true';

const datasource = mocked(_datasource);
const hostRules = mocked(_hostRules);
const artifactsExtra = mocked(_artifactsExtra);

const gomod1 = codeBlock`
module github.com/renovate-tests/gomod1
Expand Down Expand Up @@ -1866,6 +1869,47 @@ describe('modules/manager/gomod/artifacts', () => {
expect(execSnapshots).toMatchObject(expectedResult);
});

it('returns artifact notices', async () => {
artifactsExtra.getExtraDepsNotice.mockReturnValue('some extra notice');
GlobalConfig.set({ ...adminConfig, binarySource: 'docker' });
fs.readLocalFile.mockResolvedValueOnce('Current go.sum');
fs.readLocalFile.mockResolvedValueOnce(null); // vendor modules filename
fs.readLocalFile.mockResolvedValueOnce('someText\n\ngo 1.17\n\n');
mockExecAll();
git.getRepoStatus.mockResolvedValueOnce(
partial<StatusResult>({
modified: ['go.sum', 'main.go'],
}),
);
fs.readLocalFile
.mockResolvedValueOnce('New go.sum')
.mockResolvedValueOnce('New main.go')
.mockResolvedValueOnce('New go.mod');
datasource.getPkgReleases.mockResolvedValueOnce({
releases: [{ version: '1.17.0' }, { version: '1.14.0' }],
});
const res = await gomod.updateArtifacts({
packageFileName: 'go.mod',
updatedDeps: [
{ depName: 'github.com/google/go-github/v24', newVersion: 'v28.0.0' },
],
newPackageFileContent: gomod1,
config: {
updateType: 'major',
postUpdateOptions: ['gomodUpdateImportPaths'],
},
});

expect(res).toEqual([
{ file: { type: 'addition', path: 'go.sum', contents: 'New go.sum' } },
{ file: { type: 'addition', path: 'main.go', contents: 'New main.go' } },
{
file: { type: 'addition', path: 'go.mod', contents: 'New go.mod' },
notice: { file: 'go.mod', message: 'some extra notice' },
},
]);
});

it('config contains go version', async () => {
GlobalConfig.set({ ...adminConfig, binarySource: 'docker' });
fs.readLocalFile.mockResolvedValueOnce('Current go.sum');
Expand Down
24 changes: 21 additions & 3 deletions lib/modules/manager/gomod/artifacts.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { logger } from '../../../logger';
import { coerceArray } from '../../../util/array';
import { exec } from '../../../util/exec';
import type { ExecOptions } from '../../../util/exec/types';
import { filterMap } from '../../../util/filter-map';
import {
ensureCacheDir,
isValidLocalPath,
Expand All @@ -24,6 +25,7 @@ import type {
UpdateArtifactsConfig,
UpdateArtifactsResult,
} from '../types';
import { getExtraDepsNotice } from './artifacts-extra';

const { major, valid } = semver;

Expand Down Expand Up @@ -364,14 +366,30 @@ export async function updateArtifacts({
.replace(regEx(/\/\/ renovate-replace /g), '')
.replace(regEx(/renovate-replace-bracket/g), ')');
if (finalGoModContent !== newGoModContent) {
logger.debug('Found updated go.mod after go.sum update');
res.push({
const artifactResult: UpdateArtifactsResult = {
file: {
type: 'addition',
path: goModFileName,
contents: finalGoModContent,
},
});
};

const updatedDepNames = filterMap(updatedDeps, (dep) => dep?.depName);
const extraDepsNotice = getExtraDepsNotice(
newGoModContent,
finalGoModContent,
updatedDepNames,
);

if (extraDepsNotice) {
artifactResult.notice = {
file: goModFileName,
message: extraDepsNotice,
};
}

logger.debug('Found updated go.mod after go.sum update');
res.push(artifactResult);
}
return res;
} catch (err) {
Expand Down
6 changes: 6 additions & 0 deletions lib/modules/manager/gomod/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,9 @@ export interface MultiLineParseResult {
reachedLine: number;
detectedDeps: PackageDependency[];
}

export interface ExtraDep {
depName: string;
currentValue: string;
newValue: string;
}
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,7 @@
"deepmerge": "4.3.1",
"dequal": "2.0.3",
"detect-indent": "6.1.0",
"diff": "5.2.0",
"editorconfig": "2.0.0",
"email-addresses": "5.0.0",
"emoji-regex": "10.3.0",
Expand Down Expand Up @@ -312,7 +313,6 @@
"callsite": "1.0.0",
"common-tags": "1.8.2",
"conventional-changelog-conventionalcommits": "7.0.2",
"diff": "5.2.0",
"emojibase-data": "15.3.0",
"eslint": "8.57.0",
"eslint-formatter-gha": "1.4.3",
Expand Down
6 changes: 3 additions & 3 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.