test (e2e) : move manpage checks to a separate scenario (#4608)

[rohanKanojia]

Description

Fixes: #4608

Relates to: #4586

In #4586 , I had added step to check man -P cat crc to verify whether man pages are correctly generated after executing crc setup in basic scenario

• Currently, man pages are only generated for Linux/MacOS. I had added this in basic scenario which caused failure on Windows.
• On MacOS, I'm seeing an issue that if for some reason manpath is not updated correctly; man command fails to list manpages for CRC. I see that there is an already existing entry for some crc named man page (see man pages didn't cleaned after crc cleanup #4608 (comment))

Update basic.feature to remove manpage checks and add a new scenario manpages.feature to verify manpage generation and cleanup. This new scenario would only be enabled for @darwin and @linux environments.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• Feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change
• Chore (non-breaking change which doesn't affect codebase;
  test, version modification, documentation, etc.)

Proposed changes

These changes update how we verify manpage generation:

• Update manpages step in basic scenario to skip execution for windows
• Instead of relying of man command output, only verify whether we've generated the man pages files correctly in the specified directory.

Testing

I've run manpages.feature on MacOS, Linux machine to verify that these changes don't break anything.

Contribution Checklist

• I Keep It Small and Simple: The smaller the PR is, the easier it is to review and have it merged
• I have performed a self-review of my code
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Which platform have you tested the code changes on?
  • Linux
  • Windows
  • MacOS

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[lilyLuLiu]

@rohanKanojia
I've tested the case in different mac. It pass on mac-4 but fail on mac-2 machine.
The failure reason is that man can't find the man pages.
On mac-4:

manpath
/Users/podmanqe/.local/share/man:/opt/homebrew/opt/node@22/share/man:/opt/homebrew/share/man:/usr/local/share/man:/System/Cryptexes/App/usr/share/man:/usr/share/man:/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/share/man:/Library/Developer/CommandLineTools/usr/share/man

on mac-2

manpath
/usr/local/share/man:/System/Cryptexes/App/usr/share/man:/usr/share/man:/opt/X11/share/man:/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/share/man:/Applications/Xcode.app/Contents/Developer/usr/share/man:/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/share/man

So we need to add the man page location into the manpath
After running export MANPATH=/Users/crcqe/.local/share/man. The test pass.

[rohanKanojia]

@lilyLuLiu : As far as I remember mac-brno1 and mac-brno2 are amd64 machines right? And mac-brno4 is arm64?

I had code in place to set the MANPATH environment variable:

crc/pkg/crc/manpages/manpages_unix.go

Line 62 in 20e0dd2

err = appendToManPathEnvironmentVariable(targetDir)

Let me try executing export command instead.

[rohanKanojia]

@albfan @lilyLuLiu: I verified the updated version of this PR on the mac-brno2 instance. It seems to be passing. It also seems to be passing on ci which has linux runner (see logs)

go install -tags "containers_image_openpgp"  -ldflags="-X github.com/crc-org/crc/v2/pkg/crc/version.crcVersion=2.47.0 -X github.com/crc-org/crc/v2/pkg/crc/version.ocpVersion=4.17.14 -X github.com/crc-org/crc/v2/pkg/crc/version.okdVersion=4.15.0-0.okd-2024-02-23-163410 -X github.com/crc-org/crc/v2/pkg/crc/version.microshiftVersion=4.17.14 -X github.com/crc-org/crc/v2/pkg/crc/version.commitSha=50bdb6 "  ./cmd/crc
Log successfully started, logging into: /Users/crcqe/Documents/crc/test/e2e/out/test-results/e2e_2025-2-26_14-09-55.log
Running e2e test in: /Users/crcqe/Documents/crc/test/e2e/out/test-run
Working directory set to: /Users/crcqe/Documents/crc/test/e2e/out/test-run
Expecting the bundle provided by the user
Using existing bundle: /Users/crcqe/.crc/cache/crc_vfkit_4.18.1_amd64
Feature: Check generation and cleanup of manpages

  Scenario: verify man pages are generated after crc setup and deleted on cleanup # features/manpages.feature:5
    When executing single crc setup command succeeds                              # testsuite.go:954 -> github.com/crc-org/crc/v2/test/e2e/testsuite.ExecuteSingleCommandWithExpectedExitStatus
    Then accessing crc man pages succeeds                                         # testsuite.go:1079 -> github.com/crc-org/crc/v2/test/e2e/testsuite.EnsureCRCManPagesAvailability
    When executing crc cleanup command succeeds                                   # testsuite.go:947 -> github.com/crc-org/crc/v2/test/e2e/testsuite.ExecuteCommandWithExpectedExitStatus
    Then accessing crc man pages fails                                            # testsuite.go:1079 -> github.com/crc-org/crc/v2/test/e2e/testsuite.EnsureCRCManPagesAvailability
Deleted CRC instance (if one existed).

1 scenarios (1 passed)
4 steps (4 passed)
15.676635058s
ok  	github.com/crc-org/crc/v2/test/e2e	16.886s

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: lilyLuLiu
Once this PR has been reviewed and has the lgtm label, please assign gbraad for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• test/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@rohanKanojia: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-crc	21448e5	link	true	/test e2e-crc

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.