[ci] Add debian-12 to matrix in packaging and platform jobs #116172

jozala · 2024-11-04T12:52:07Z

No description provided.

jozala · 2024-11-21T12:12:22Z

distribution/packages/build.gradle

@@ -335,7 +335,6 @@ Closure commonDebConfig(String architecture) {

    // versions found on oldest supported distro, centos-6
    requires('bash', '4.1', GREATER | EQUAL)
-    requires('lsb-base', '4', GREATER | EQUAL)


Library lsb-base is now abandoned and marked as obsolete by Lintian. This lib is not available in Debian 12.
As far as I understand it can be used for some logging function, but I couldn't find any direct calls to it from ES code.
The tests has passed on all DEB-based distributions so I assume it is safe to delete it.

Presumable it's just because it brings in sysvinit-utils. @rjernst do you recall why this might be needed?

jozala · 2024-11-21T12:16:14Z

qa/packaging/src/test/java/org/elasticsearch/packaging/test/DebMetadataTests.java

+        Shell.Result result = sh.runIgnoreExitCode(
+            String.format(Locale.ROOT, "lintian %s %s", extraArgs, getDistributionFile(distribution()))
+        );
+        Result lintianResult = lintianParser.parse(result.stdout());
+        // Unfortunately Lintian overrides syntax changes between Lintian versions in a non-backwards compatible
+        // way, so we have to manage some exclusions outside the overrides file.
+        if (lintianResult.isSuccess() == false) {
+            List<Issue> importantIssues = lintianResult.issues()
+                .stream()
+                .filter(issue -> IGNORED_TAGS.contains(issue.tag()) == false)
+                .toList();
+            if (importantIssues.isEmpty() == false) {
+                fail(
+                    "Issues for DEB package found by Lintian:\n"
+                        + importantIssues.stream().map(Record::toString).collect(Collectors.joining("\n"))
+                );


Instead of using exit code, the Lintian output is now parsed and some tags are ignored. It is needed, because in the newer version of Lintian tag mismatched-override is a "Show-Always" tag which means it cannot be suppressed.

jozala · 2024-11-21T12:18:27Z

qa/packaging/src/test/java/org/elasticsearch/packaging/test/DebMetadataTests.java

-            // way, so we have to tolerate these (or maintain separate override files per lintian version).
-            if (helpText.contains("--suppress-tags")) {
-                extraArgs += " --suppress-tags malformed-override,mismatched-override";
+            extraArgs = "--fail-on error,warning";


Lintian --fail-on requires all levels, to fail on, to be listed .

'warning' does not imply 'error'
https://manpages.debian.org/testing/lintian/lintian.1.en.html

Lintian test has been changed to parse the result instead of using exit code. This was required, because now `mismatched-override` is non-erasable tag which cannot be ignored for exit code. Lintian introduced non-backward-compatible format change for overrides file. Because of that, some overrides are now duplicated in a format for older versions. Additionally, Lintian overrides file has been cleaned up to remove the tags which are no longer failing.

Replaced the usage of `InetAddress.getLocalHost()` with `InetAddress.getLoopbackAddress()` in the `testSlowNodeCanBeCancelled` test to make it work on Debian 12 where `InetAddress.getLocalHost()` may return non-existing address due to incorrectly set value in `/etc/hosts` by GCP VM script `google_set_hostname`.

Removed redundant code that checks for specific exit codes when running Lintian. Different versions of Lintian produces different exit codes it is impossible to relay on them.

This reverts commit 4d9bf31.

jozala · 2024-11-21T12:24:18Z

Periodic builds for this branch passed:

elasticsearch / periodic / packaging
elasticsearch / periodic / platform-support - failing for rhel-9 and rocky-9, but the same failures are on main so it seems unrelated

elasticsearchmachine · 2024-11-21T12:25:55Z

Pinging @elastic/es-delivery (Team:Delivery)

mark-vieira · 2024-11-21T20:21:15Z

qa/packaging/src/test/java/org/elasticsearch/packaging/util/LintianResultParser.java

+public class LintianResultParser {
+
+    private static final Logger logger = LogManager.getLogger(LintianResultParser.class);
+    private static final Pattern RESULT_PATTERN = Pattern.compile("(?<severity>[EW]): (?<package>\\S+): (?<tag>\\S+) (?<message>.+)");


I don't think I've ever used "named" regex capture groups but I think I will going forward 👍

elasticsearchmachine · 2024-11-22T09:04:19Z

💔 Backport failed

Status	Branch	Result
❌	7.17	Commit could not be cherrypicked due to conflicts
✅	8.x

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 116172

…116172) Lintian test has been changed to parse the result instead of using exit code. This was required, because now `mismatched-override` is non-erasable tag which cannot be ignored for exit code. Lintian introduced non-backward-compatible format change for overrides file. Because of that, some overrides are now duplicated in a format for older versions. Additionally, Lintian overrides file has been cleaned up to remove the tags which are no longer failing.

…#117315) Lintian test has been changed to parse the result instead of using exit code. This was required, because now `mismatched-override` is non-erasable tag which cannot be ignored for exit code. Lintian introduced non-backward-compatible format change for overrides file. Because of that, some overrides are now duplicated in a format for older versions. Additionally, Lintian overrides file has been cleaned up to remove the tags which are no longer failing.

jozala · 2024-11-22T12:39:42Z

💚 All backports created successfully

Status	Branch	Result
✅	7.17

Questions ?

Please refer to the Backport tool documentation

Lintian test has been changed to parse the result instead of using exit code. This was required, because now `mismatched-override` is non-erasable tag which cannot be ignored for exit code. Lintian introduced non-backward-compatible format change for overrides file. Because of that, some overrides are now duplicated in a format for older versions. Additionally, Lintian overrides file has been cleaned up to remove the tags which are no longer failing. (cherry picked from commit 2ac267d) # Conflicts: # distribution/packages/src/deb/lintian/elasticsearch # qa/os/src/test/java/org/elasticsearch/packaging/util/LintianResultParser.java # qa/packaging/src/test/java/org/elasticsearch/packaging/test/DebMetadataTests.java

…116172) Lintian test has been changed to parse the result instead of using exit code. This was required, because now `mismatched-override` is non-erasable tag which cannot be ignored for exit code. Lintian introduced non-backward-compatible format change for overrides file. Because of that, some overrides are now duplicated in a format for older versions. Additionally, Lintian overrides file has been cleaned up to remove the tags which are no longer failing.

…116172) (#117336) * [ci] Add debian-12 to matrix in packaging and platform jobs (#116172) Lintian test has been changed to parse the result instead of using exit code. This was required, because now `mismatched-override` is non-erasable tag which cannot be ignored for exit code. Lintian introduced non-backward-compatible format change for overrides file. Because of that, some overrides are now duplicated in a format for older versions. Additionally, Lintian overrides file has been cleaned up to remove the tags which are no longer failing. (cherry picked from commit 2ac267d) # Conflicts: # distribution/packages/src/deb/lintian/elasticsearch # qa/os/src/test/java/org/elasticsearch/packaging/util/LintianResultParser.java # qa/packaging/src/test/java/org/elasticsearch/packaging/test/DebMetadataTests.java * Spotless fix

…116172) Lintian test has been changed to parse the result instead of using exit code. This was required, because now `mismatched-override` is non-erasable tag which cannot be ignored for exit code. Lintian introduced non-backward-compatible format change for overrides file. Because of that, some overrides are now duplicated in a format for older versions. Additionally, Lintian overrides file has been cleaned up to remove the tags which are no longer failing.

jozala added >non-issue :Delivery/Build Build or test infrastructure auto-backport Automatically create backport pull requests when merged v8.17.0 labels Nov 4, 2024

elasticsearchmachine added the v9.0.0 label Nov 4, 2024

jozala force-pushed the ci-pipelines-add-debian-12 branch from 0aba1c7 to 52af548 Compare November 13, 2024 10:01

jozala force-pushed the ci-pipelines-add-debian-12 branch from cd5a1ce to 1ec74c6 Compare November 20, 2024 14:39

elasticsearchmachine added v8.18.0 and removed v8.17.0 labels Nov 20, 2024

jozala commented Nov 21, 2024

View reviewed changes

jozala added 6 commits November 21, 2024 13:20

[ci] Add debian-12 to matrix in packaging and platform jobs

7fbc1ac

[ci] Regenerating the periodic-packaging.yml from template

a122f0a

Remove Lintian exit code check in test

1e09b40

Removed redundant code that checks for specific exit codes when running Lintian. Different versions of Lintian produces different exit codes it is impossible to relay on them.

Revert "Use loopback address in RemoteClusterConnectionTests"

d3b0365

This reverts commit 4d9bf31.

jozala force-pushed the ci-pipelines-add-debian-12 branch from 5952df2 to d3b0365 Compare November 21, 2024 12:20

jozala marked this pull request as ready for review November 21, 2024 12:25

jozala requested a review from a team as a code owner November 21, 2024 12:25

elasticsearchmachine added the Team:Delivery Meta label for Delivery team label Nov 21, 2024

jozala requested a review from breskeby November 21, 2024 13:17

jozala added the v7.17.26 label Nov 21, 2024

mark-vieira approved these changes Nov 21, 2024

View reviewed changes

jozala merged commit 2ac267d into elastic:main Nov 22, 2024
16 checks passed

jozala mentioned this pull request Nov 22, 2024

[8.x] [ci] Add debian-12 to matrix in packaging and platform jobs (#116172) #117315

Merged

elasticsearchmachine added the backport pending label Nov 22, 2024

jozala mentioned this pull request Nov 22, 2024

[7.17] [ci] Add debian-12 to matrix in packaging and platform jobs (#116172) #117336

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] Add debian-12 to matrix in packaging and platform jobs #116172

[ci] Add debian-12 to matrix in packaging and platform jobs #116172

jozala commented Nov 4, 2024

jozala Nov 21, 2024

mark-vieira Nov 21, 2024

jozala Nov 21, 2024

jozala Nov 21, 2024

jozala commented Nov 21, 2024

elasticsearchmachine commented Nov 21, 2024

mark-vieira Nov 21, 2024

elasticsearchmachine commented Nov 22, 2024

jozala commented Nov 22, 2024

[ci] Add debian-12 to matrix in packaging and platform jobs #116172

[ci] Add debian-12 to matrix in packaging and platform jobs #116172

Conversation

jozala commented Nov 4, 2024

jozala Nov 21, 2024

Choose a reason for hiding this comment

mark-vieira Nov 21, 2024

Choose a reason for hiding this comment

jozala Nov 21, 2024

Choose a reason for hiding this comment

jozala Nov 21, 2024

Choose a reason for hiding this comment

jozala commented Nov 21, 2024

elasticsearchmachine commented Nov 21, 2024

mark-vieira Nov 21, 2024

Choose a reason for hiding this comment

elasticsearchmachine commented Nov 22, 2024

💔 Backport failed

jozala commented Nov 22, 2024

💚 All backports created successfully

Questions ?