Skip to content

Conversation

@jorgee
Copy link
Contributor

@jorgee jorgee commented Nov 18, 2024

close #5470

Adds the CmdEvalParams in the task outputs as a key to compute the hash

@jorgee jorgee linked an issue Nov 18, 2024 that may be closed by this pull request
@netlify
Copy link

netlify bot commented Nov 18, 2024

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit 3ec20f0
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/68879e7b3b38c100083980bd

@jorgee jorgee force-pushed the 5470-pipeline-uses-cached-process-despite-edited-eval-script branch from 40ecdc4 to d605b2b Compare November 18, 2024 13:44
@bentsherman bentsherman added the cache-breaking Changes that will break everyone's task cache label Dec 13, 2024
@jorgee jorgee requested a review from pditommaso January 30, 2025 19:29
@pditommaso
Copy link
Member

This alter the assumption the task is idempotent. The task hash should only rely on the inputs not on the outputs

@pditommaso
Copy link
Member

I see the point it could be considered as part of the execution script, but still in principle it should not be needed to re-run the task

@bentsherman
Copy link
Member

Considering that eval is a shorthand for export an output env var in the script, it is essentially part of the script and therefore part of the task identity.

I think it is possible to evaluate the task outputs without "resolving" or "collecting" them, before computing the task hash. Then you can see if any output evals were declared and include them as if they were part of the script

@pditommaso pditommaso force-pushed the master branch 2 times, most recently from 5a93547 to 27345a6 Compare February 10, 2025 21:46
@pditommaso pditommaso force-pushed the master branch 3 times, most recently from b4b321e to 069653d Compare June 4, 2025 18:54
@bentsherman bentsherman requested review from bentsherman and removed request for pditommaso July 28, 2025 15:50
@bentsherman bentsherman force-pushed the 5470-pipeline-uses-cached-process-despite-edited-eval-script branch from 7971fc1 to af3d7bc Compare July 28, 2025 15:58
Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@bentsherman bentsherman force-pushed the 5470-pipeline-uses-cached-process-despite-edited-eval-script branch from af3d7bc to 3ec20f0 Compare July 28, 2025 15:59
@bentsherman bentsherman merged commit b0fe0a9 into master Jul 28, 2025
22 checks passed
@bentsherman bentsherman deleted the 5470-pipeline-uses-cached-process-despite-edited-eval-script branch July 28, 2025 16:40
@pditommaso
Copy link
Member

This needs to be marked as a cache breaking change in the changelog

@pditommaso
Copy link
Member

also I really not sure about this, because break the contract that task ID depends only on inputs + script

@bentsherman
Copy link
Member

The eval commands are part of the script

pditommaso added a commit that referenced this pull request Aug 15, 2025
This reverts commit b0fe0a9.

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
pditommaso added a commit that referenced this pull request Aug 15, 2025
This reverts commit b0fe0a9.

Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cache-breaking Changes that will break everyone's task cache

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pipeline uses cached process despite edited eval script

4 participants