Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#209 When "dir:" attribute points to a non-existing dir, create it #211

Merged
merged 6 commits into from
Jun 16, 2019

Conversation

marco-m
Copy link
Contributor

@marco-m marco-m commented Jun 4, 2019

Also:

  • Task directory: test when "dir:" attribute points to an existing dir
  • Task directory: test default case (no "dir:" attribute)
  • FIx spelling

Closes #209

@marco-m
Copy link
Contributor Author

marco-m commented Jun 6, 2019

Ah, I didn't realize that CI failed.

@marco-m
Copy link
Contributor Author

marco-m commented Jun 6, 2019

@andreynering I fixed the embarrassing CI failure ;-)
This is now ready for review.

task.go Outdated
// If so, we create it.
if t.Dir != "" {
if _, err := os.Stat(t.Dir); os.IsNotExist(err) {
if err := os.MkdirAll(t.Dir, 0755); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps move this code to Executor.Setup()?

That would prevent race conditions if the same task is ran in parallel.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhh, I am not very familiar with the internals of task, but here is what I found:

  • To create the directory, we need attribute t.Dir. At the very beginning of RunTask(), we call CompiledTask()

    task/task.go

    Line 178 in 9c475c3

    t, err := e.CompiledTask(call)
    which can also change the value of t.Dir

If we were to move this code to Setup(), we would have to do the following:

  • Add a loop that goes through all the tasks, so that it can look at each call of type taskfile.Call, call t := CompiledTask(call) and then create the directory based on t.Dir ? This would mean also that it would create directories also for tasks that are not required on the command line (directly or indirectly via dependencies).

If my understanding is right, then I need some more guidance on how to do this properly :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was also thinking: if there is a race condition in calling RunTask(), then this race condition is also there independently from this PR, or am I missing something?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @marco-m,

You're right that it'd be more complicated. I totally forgot that we'd need to compile each task.

I think we can then just have a mutex/lock for that. Extracting into another function would be nice, too.

func (t *Task) mkdir() error {
	if t.Dir == "" {
		return nil
	}

	t.mkdirMutex.Lock()
	defer t.mkdirMutex.Unlock()

	// ...
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andreynering I can do that. But what I am missing is this: the mutex would protect for the specific race when creating a directory, but it looks like there is an inherent race when task run in parallel? Wouldn't make more sense to put the mutex outside this new mkdir() function ?

@marco-m
Copy link
Contributor Author

marco-m commented Jun 10, 2019

@andreynering I added the mutex, but please see my questions above about the inherent race.

@marco-m
Copy link
Contributor Author

marco-m commented Jun 13, 2019

ping

@andreynering andreynering merged commit 733c563 into go-task:master Jun 16, 2019
andreynering added a commit that referenced this pull request Jun 16, 2019
@andreynering
Copy link
Member

Hi @marco-m,

This is merged. I did some small changes on fe2b8c8.

About your question above: tasks are supposed to run in parallel, and there's no race condition in Task itself, unless the user programs a task that won't work if ran in parallel.

But since the dir creation is Task work, let's keep the mutex to keep things safe. 🙂

@marco-m
Copy link
Contributor Author

marco-m commented Jun 16, 2019

Thank you!

@marco-m
Copy link
Contributor Author

marco-m commented Jun 16, 2019

I see you moved the mutex code. Yes, I was unsure where to put it.

marco-m added a commit to marco-m/task that referenced this pull request Jun 16, 2019
andreynering added a commit that referenced this pull request Jun 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Task directory: allow to create directory if missing
2 participants