Skip to content

Conversation

@justinvreeland
Copy link
Contributor

@justinvreeland justinvreeland commented Oct 9, 2025

I want this to be for development not for building which is why i'm not doing what sources did and running the pipeline which should be fine shell code. I also want all of the history if i'm doing development.

IMO Ideally the source locations would be used in pipelines but not actually defined in it so that other pipelines could reference it and melange could make smart choices. The update section for example of one that already works around this.

I'd really like to make a world in the future where we can enforce that all patches are applied in a sensible predictable order independent of the melange tool. As far as I can tell right now the best way to do that is to apply them to the source an pop ALL of our patches off as a stack from format-patch.

@justinvreeland
Copy link
Contributor Author

build this and then run something like:

/Users/justin/src/chainguard-dev/melange/melange source -o sources dash.yaml

@justinvreeland
Copy link
Contributor Author

justin@justins-MacBook-Pro ~/src/wolfi-dev/wolfi ((no description set) rmwo c2) 
❯ /Users/justin/src/chainguard-dev/melange/melange source get -o sources apk-tools.yaml
2025/10/17 08:27:45 INFO Found git-checkout step
2025/10/17 08:27:45 INFO Cloning https://gitlab.alpinelinux.org/alpine/apk-tools.git to sources/apk-tools
Enumerating objects: 15140, done.
Counting objects: 100% (188/188), done.
Compressing objects: 100% (114/114), done.
Total 15140 (delta 104), reused 133 (delta 73), pack-reused 14952 (from 1)
2025/10/17 08:27:49 INFO Checking out commit 9d074efdc12bc41b5d24190595a5269a770e852a
2025/10/17 08:27:49 INFO Checked out commit 9d074efdc12bc41b5d24190595a5269a770e852a
2025/10/17 08:27:49 INFO Applying patches
2025/10/17 08:27:49 INFO Applying patch /Users/justin/src/wolfi-dev/wolfi/apk-tools/292.patch
Applying: db: Explicitly clean st on fstatat failure to workaround rosetta2
Your browser will now be opened to:
https://oauth2.sigstore.dev/auth/auth?access_type=online&client_id=sigstore&code_challenge=9TX77mG_-KLDvOze94yC524uqm2Db5aQEYUx2stiNzo&code_challenge_method=S256&connector_id=https%3A%2F%2Faccounts.google.com&nonce=34CQ2x4jciTa6zbkuDa7WIabJlL&redirect_uri=http%3A%2F%2Flocalhost%3A57070%2Fauth%2Fcallback&response_type=code&scope=openid+email&state=34CQ2xqxGCI9cNXfnRYvbJwsVnJ
tlog entry created with index: 619144277
2025/10/17 08:27:51 INFO Applying patch /Users/justin/src/wolfi-dev/wolfi/apk-tools/usrmerge-lib.patch
Patch format detection failed.
2025/10/17 08:27:52 INFO git am failed, aborting to clean up
2025/10/17 08:27:52 INFO Checking if patch can be applied with git apply
Your browser will now be opened to:
https://oauth2.sigstore.dev/auth/auth?access_type=online&client_id=sigstore&code_challenge=WkBHVmDJMkGeVpKKsr_Ynb8xBdWWuBCEas1AzNuM0-U&code_challenge_method=S256&connector_id=https%3A%2F%2Faccounts.google.com&nonce=34CQ3FxVohulJiFsZUmxRmYiA9x&redirect_uri=http%3A%2F%2Flocalhost%3A57083%2Fauth%2Fcallback&response_type=code&scope=openid+email&state=34CQ3GnYqTZ6xVPSDi9Hn8SQuqR
tlog entry created with index: 619144494
[detached HEAD 0b6b45f] Apply patch: usrmerge-lib.patch
 1 file changed, 6 insertions(+)
2025/10/17 08:27:53 INFO Applied patch /Users/justin/src/wolfi-dev/wolfi/apk-tools/usrmerge-lib.patch using git apply + commit
2025/10/17 08:27:53 INFO Applying patch /Users/justin/src/wolfi-dev/wolfi/apk-tools/0001-Work-without-fips-sha1.patch
Applying: Work without fips sha1
Your browser will now be opened to:
https://oauth2.sigstore.dev/auth/auth?access_type=online&client_id=sigstore&code_challenge=g7tozBl590VV5_gZOL42lcKLC_WOjcfobgasHiV5XW8&code_challenge_method=S256&connector_id=https%3A%2F%2Faccounts.google.com&nonce=34CQ3UYGuy9rAsAkYMk0MLS6kih&redirect_uri=http%3A%2F%2Flocalhost%3A57089%2Fauth%2Fcallback&response_type=code&scope=openid+email&state=34CQ3SPDyE2lu1V4U2vzjueO05m
tlog entry created with index: 619144672
2025/10/17 08:27:55 INFO Successfully extracted source to sources

 justin@justins-MacBook-Pro ~/src/wolfi-dev/wolfi ((no description set) rmwo 87f) 
❯ /Users/justin/src/chainguard-dev/melange/melange source pop -o sources apk-tools.yaml
2025/10/17 08:28:01 INFO Generating patches from 9d074efdc12bc41b5d24190595a5269a770e852a in /Users/justin/src/wolfi-dev/wolfi/sources/apk-tools
2025/10/17 08:28:02 INFO Generated 3 patches: [0001-db-Explicitly-clean-st-on-fstatat-failure-to-workaro.patch 0002-Apply-patch-usrmerge-lib.patch.patch 0003-Work-without-fips-sha1.patch]
2025/10/17 08:28:02 INFO Formatted YAML with yam
2025/10/17 08:28:02 INFO Updated apk-tools.yaml with git-am pipeline using 3 patches

 justin@justins-MacBook-Pro ~/src/wolfi-dev/wolfi ((no description set) rmwo 87f) 
❯ jj diff apk-tools.yaml
Error: Failed to reset Git HEAD state
Caused by:
1: Could not acquire lock for index file
2: The lock for resource '/Users/justin/src/wolfi-dev/wolfi/.git/index' could not be obtained immediately after 1 attempt(s). The lockfile at '/Users/justin/src/wolfi-dev/wolfi/.git/index.lock' might need manual deletion.

 justin@justins-MacBook-Pro ~/src/wolfi-dev/wolfi ((no description set) rmwo 87f) 
[1]❯ rm /Users/justin/src/wolfi-dev/wolfi/.git/index

 justin@justins-MacBook-Pro ~/src/wolfi-dev/wolfi ((no description set) rmwo 87f) 
❯ jj diff apk-tools.yaml
Error: Failed to reset Git HEAD state
Caused by:
1: Could not acquire lock for index file
2: The lock for resource '/Users/justin/src/wolfi-dev/wolfi/.git/index' could not be obtained immediately after 1 attempt(s). The lockfile at '/Users/justin/src/wolfi-dev/wolfi/.git/index.lock' might need manual deletion.

 justin@justins-MacBook-Pro ~/src/wolfi-dev/wolfi ((no description set) rmwo 87f) 
[1]❯ rm /Users/justin/src/wolfi-dev/wolfi/.git/index.lock

 justin@justins-MacBook-Pro ~/src/wolfi-dev/wolfi ((no description set) rmwo 87f) 
❯ jj diff apk-tools.yaml
Modified regular file apk-tools.yaml:
    ...
  31   31:       tag: v${{package.version}}
  32   32:       expected-commit: 9d074efdc12bc41b5d24190595a5269a770e852a
  33   33: 
  34   34:   - uses: patchgit-am
  35   35:     with:
  36     :       patches: 292.patch usrmerge-lib.patch 0001-Work-without-fips-sha1.patch
       36:       patches: 0001-db-Explicitly-clean-st-on-fstatat-failure-to-workaro.patch 0002-Apply-patch-usrmerge-lib.patch.patch 0003-Work-without-fips-sha1.patch
  37   37: 
  38   38:   - runs: |
  39   39:       sed -i -e 's:-Werror::' Make.rules
    ...

 justin@justins-MacBook-Pro ~/src/wolfi-dev/wolfi ((no description set) rmwo 9727) 
❯ jj st apk-tools
Working copy changes:
A apk-tools/0001-db-Explicitly-clean-st-on-fstatat-failure-to-workaro.patch
A apk-tools/0002-Apply-patch-usrmerge-lib.patch.patch
A apk-tools/0003-Work-without-fips-sha1.patch
Working copy  (@) : rmwoyrqy 972735aa (no description set)
Parent commit (@-): qqtoltyo df93563b a45af8da push-qqtoltyovvlk | aws*/s2n: Bump versions to avoid issues with FIPS provide priority

@justinvreeland
Copy link
Contributor Author

@sergiodj Take a look at the new updates.

@justinvreeland
Copy link
Contributor Author

I feel like i could probably just edit the melange config and write it out somehow and not call yam but this is mostly just poc stuff.

@justinvreeland justinvreeland changed the title DNM DRAFT: Source fetch from melange RFC: Source fetch from melange Oct 30, 2025
This subcommand pulls the source a build config. Currently it only
supports git-checkout but in the future it should suppport more
functionality.

The purpose of this is to make interacting with upstream source and
patches easier.  I'd like a workflow like `melange source -o sources/
package.yaml` to setup a git tree with patches applied and another
command that can do the revese.

There's something similiar in source.go for license checks. That version
runs the same shell script that the pipeline has.  This command is meant
for setting up an environment that can be used to develope against the
upstream sources. I'm not interested shallow cloning or keeping a
consistent commiter id or anything here. I'm realy just trying to make
it easy to develop/debug packages using the upstream source.
@justinvreeland justinvreeland marked this pull request as ready for review October 30, 2025 22:31
@justinvreeland
Copy link
Contributor Author

2025/10/30 22:34:57 INFO running step "fetch"
2025/10/30 22:34:57 WARN --2025-10-30 22:34:57--  https://ftpmirror.gnu.org/gnu/hello/hello-2.12.2.tar.gz
2025/10/30 22:34:57 WARN Resolving ftpmirror.gnu.org... 209.51.188.200, 2001:470:142:5::200
2025/10/30 22:34:59 WARN Connecting to ftpmirror.gnu.org|209.51.188.200|:443... connected.
2025/10/30 22:34:59 WARN HTTP request sent, awaiting response... 502 Bad Gateway
2025/10/30 22:34:59 WARN 2025-10-30 22:34:59 ERROR 502: Bad Gateway.
2025/10/30 22:34:59 WARN
2025/10/30 22:35:00 ERRO failed to build package: unable to run package hello-wolfi pipeline: unable to run pipeline: unable to run pipeline: exit status 8
make[1]: *** [Makefile:151: packages/x86_64/hello-wolfi-2.12.2-r3.apk] Error 1
make[1]: Leaving directory '/home/runner/work/melange/melange'

of course it is.

@justinvreeland
Copy link
Contributor Author

@sil2100 we might want to merge these in some way #1979

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants