[go: nahoru, domu]

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

git checkout of pipeline can be changed mid-execution #2655

Closed
notestaff opened this issue Feb 17, 2022 · 5 comments
Closed

git checkout of pipeline can be changed mid-execution #2655

notestaff opened this issue Feb 17, 2022 · 5 comments

Comments

@notestaff
Copy link

Bug report

If I start a nextflow run on a particular git revision of a pipeline, and then, while this run is executing, do a nextflow pull or start another nextflow run on a different revision, the pipeline checkout used by the still-executing original run (under ~/.nextflow/assets/scm) will be overwritten, possibly leading to wrong results. Instead, nextflow should create a separate checkout for each git revision (perhaps using git worktree) for which a pipeline run is started, and keep that checkout immutable throughout the pipeline's run.

@pditommaso
Copy link
Member

This is a good point. Ideally, this should require having Nextlow to clone the repo as a bare repository.

Then, when the execution is launched it should check it out into a sub-directory named as the requested versions

@leonorpalmeira
Copy link

I would have also expected main.nf to be read once and kept into RAM (or copied on disk into the execution directory) while nextflow executes the pipeline. Instead, I experience the same issue as above in the following use case:

cd MyGitRepo/
git checkout master
nextflow main.nf
git checkout newFeatureBranch
nextflow main.nf
git checkout otherFeatureBranch

The execution of the sub-processes will execute the state in which the code is at the time where those processes are run, not the state in which they were when the pipeline was launched. I may be wrong but it seems that the only solution is to have one working repository per execution?

@stale
Copy link
stale bot commented Mar 18, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Mar 18, 2023
@leonorpalmeira
Copy link

Solving this Issue would be very useful for our team, as this behavior requires being extra careful when developing and running tests on development pipelines.

@stale stale bot closed this as completed Jun 10, 2023
@pditommaso pditommaso reopened this Jun 12, 2023
@stale stale bot closed this as completed Aug 12, 2023
@bentsherman
Copy link
Member
bentsherman commented Aug 12, 2023

Seems to be covered by #2870 which is in our next release milestone

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants