Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unusable asset filenames in releases #1131

Open
Paebbels opened this issue Jan 12, 2025 · 5 comments
Open

Unusable asset filenames in releases #1131

Paebbels opened this issue Jan 12, 2025 · 5 comments

Comments

@Paebbels
Copy link

The provided assets in the releases can't be used in other CI pipelines e.g. at GitHub, because the file name contain "salts".

Salts are:

  • the file name contains the commit date
  • the file name contains the commit hash

With such parts in the filename, the filename and especially the asset download URL can't be automatically composed without human interaction (and additionally researching the Git history). As a result, CI pipeline need to use a fixed URL once added by a human. Afterwards, it's hard to update and maintain these URLs.

Suggested solutions:

  1. Remove commit date and commit hash from filenames.
    Please note, assets in GitHub can have a filename and title, which don't need to match.
    In addition, a release is linked to a tag and a tag is linked to a commit. So traceability is provided. If traceability is needed to the users client PC, then the information should be preserved inside the file as an attestation, because files can also be renamed on download.

  2. Provide a meta file with each release describing assets as a machine readable information.
    I'll add such a file for GHDL.

  3. Provide a setup-nvc GitHub Action.
    See https://github.com/ghdl/setup-ghdl for reference.
    It could be a copy&paste&adjust for NVC, as it's not so different - in many cases even simpler than GHDL.

    Note, also the setup-ghdl Action will soon use a JSON file provided by every tagged and nightly release, because I don't want to maintain the current latest and nightly version number in parallel as hidden code inside the Action.


This problem was discovered when reworking and updating NVC for OSVVM.

/cc @JimLewis


A draft for the JSON might look like this JSON snippet (GHDL specific due to different backends):

{
  "version": "1.0",
  "meta": {
    "version": "5.0.0-dev",
    "git-hash": ""
  },
  "info": {
    "order": ["os-name", "os-version", "os-arch", "runtime","ghdl-backend"]
  },
  "ubuntu": {
    "24.04": {
      "x86-64": {
        "native": {
          "mcode": "",
          "llvm": "",
          "llvm-jit": "",
          "gcc": ""
        }
      }
    }
  },
  "macos": {
    "x86-64": {
      "13": {
        "native": {
          "mcode": ""
        }
      }
    },
    "aarch64": {
      "14": {
        "native": {
          "llvm": "",
          "llvm-jit": ""
        }
      }
    }
  },
  "windows" : {
    "x86-64": {
      "2022": {
        "native": {
          "mcode": ""
        },
        "mingw64": {
          "mcode": "",
          "llvm": ""
        },
        "ucrt64": {
          "mcode": "",
          "llvm": ""
        }
      }
    }
  }
}

Individual nodes can be queries by jq.

@nickg
Copy link
Owner

nickg commented Jan 12, 2025

Provide a setup-nvc GitHub Action.

This exists already, see nickg/setup-nvc-ci. I expect OSVVM's CI could use that directly.

@Paebbels
Copy link
Author

This exists already, see nickg/setup-nvc-ci.

Oh, my fault. I overlooked it.
When checking https://github.com/nickg, I see you're using 2 out of 6 pinned repositories. Maybe you want to consider setup-nvc-ci besides nvc in that view.

I expect OSVVM's CI could use that directly.

I did some investigations and checks and found these results:

  • The installation in MSYS2 (using the *.pkg.tar.zst packages) e.g. for UCRT64 is not supported.
    Are there plans for supporting it?
  • Installation on Windows (native) is not an option for OSVVM, because it needs a TCL interpreter like provided by MSYS2/UCRT64.
  • Reaching outside of MSYS2 is quite strange and may result in / vs. \ problems.
  • At next I'll test Ubuntu 24.04 support.

As a general note: Writing and maintaining an Action in Javascript is far more complicated then a few lines of Shell code (also due to Node.js version bumps, Javascript dependency bumps, etc.) Please have a look at setup-ghdl: https://github.com/ghdl/setup-ghdl/blob/main/action.yml. Maybe such an approach could also simplify your Action. Even if not, feel free to investigate how I solved all 5 platforms: Ubuntu, macOS Intel, macOS ARM, Windows, Windows+MSYS2.

@nickg
Copy link
Owner

nickg commented Jan 13, 2025

When checking https://github.com/nickg, I see you're using 2 out of 6 pinned repositories. Maybe you want to consider setup-nvc-ci besides nvc in that view.

Yeah that's a fair point, thanks.

The installation in MSYS2 (using the *.pkg.tar.zst packages) e.g. for UCRT64 is not supported. Are there plans for supporting it?

No, and I actually want to get rid of the binary MSYS2 packages completely now that we have the MSI installer. The problem with distributing MSYS2 packages outside of their repository is that it's a rolling distribution so dependencies quickly become stale (LLVM is the main problem).

Installation on Windows (native) is not an option for OSVVM, because it needs a TCL interpreter like provided by MSYS2/UCRT64.

The Windows build includes an embedded TCL interpreter so you can run scripts with nvc --do script.tcl. It almost works with the OSVMM build scripts except that they require tcllib which I'm not bundling.

There shouldn't be any problem calling the native Windows binary from the MSYS2 shell though. I do exactly that in a weekly job that tests the OSVVM dev branch:

https://github.com/nickg/nvc/actions/workflows/test-osvvm.yml
https://github.com/nickg/nvc/blob/master/.github/workflows/test-osvvm.yml

Reaching outside of MSYS2 is quite strange and may result in / vs. \ problems.

Windows APIs, perhaps with the exception of a few old MSCVRT ones, treat / identically to \. For example:

https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea#parameters

You may use either forward slashes (/) or backslashes () in this name.

Most of the problems I've seen come from the Unix->Windows path name translation that the MSYS2 shell does, but Mingw/UCRT binaries (i.e. not ones using the MSYS Cygwin environment) behave identically to native Windows programs in this regard.

@Paebbels
Copy link
Author

The Windows build includes an embedded TCL interpreter so you can run scripts with nvc --do script.tcl. It almost works with the OSVMM build scripts except that they require tcllib which I'm not bundling.

Is there a way to add tcllib or ship it with NVC?

OSVVM uses tcllib e.g. for YAML processing. Currently all commercial simulators have at least that library bundled.

No, and I actually want to get rid of the binary MSYS2 packages completely now that we have the MSI installer.

That could be interesting for GHDL too. We have a bad history with Windows installers due to lots of bugs in the generated MSI files... We then switched to simple ZIP files.
What tool to you use to compile the MSI?
What files / directories should I look at?

@nickg
Copy link
Owner

nickg commented Jan 13, 2025

What tool to you use to compile the MSI?

WiX.

What files / directories should I look at?

It's all under contrib/msi. I used a wizard to generate the initial XML but there's now a Python script that generates some of it automatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants