Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run Taskfiles from sub/child directories #920

Merged
merged 5 commits into from
Dec 6, 2022

Conversation

pd93
Copy link
Member

@pd93 pd93 commented Oct 31, 2022

This PR is a continuation of #289 and #408. In short, it adds support for running a Taskfile from a child/sub directory. I've illustrated some of the interesting problems this can solve below.

Problem

If we have the following directory structure:

./
├── services/
│   ├── my-service-1/
│   │   └── docker-compose.yml
│   ├── my-service-2/
│   │   └── docker-compose.yml
│   ...
│   └── my-service-n/
│       └── docker-compose.yml
└── Taskfile.yml

... and I want to create a task that can bring up each service, currently I must do one of the following:

  1. Create multiple tasks in Taskfile.yml (one for each service)
# Taskfile.yml
tasks:
  my-service-1-up:
    dir: services/my-service-1
    cmds:
      - docker-compose up -d

  # Repeat for each service...
  1. Create a Taskfile in each service directory
# services/my-service-1/Taskfile.yml
tasks:
  up:
    cmds:
      - docker-compose up -d

# Run by either cd'ing to each dir, or including the Taskfiles in the parent Taskfile.
  1. Create a reusable task and pass the directories in
# Taskfile.yml
tasks:
  up:
    dir: {{.SERVICE_DIR}}
    cmds:
      - docker-compose up -d

  my-service-1-up:
    cmds:
      - task: up
        vars: { SERVICE_DIR: "service/my-service-1" }

  # Repeat for each service...

None of these are ideal and all of them require a large amount of boilerplate code for something very simple.

Solution

This PR allows you to call task from a child directory. This, by itself, is a really nice feature. Suppose I've been running some commands in the testdata subdirectory and I want to run task test. This is now possible without having to cd ../. This functionality works the same way that Git does. It will walk up the file tree until it finds a directory containing a valid Taskfile or until it reaches the system root.

However, this functionality also offers some really creative solutions to the above problem. Using the same example as above, I can now do the following:

# Taskfile.yml
tasks:
  up:
    dir: '{{.WORKING_DIR}}'
    preconditions:
      - test -f docker-compose.yml
    cmds:
      - docker-compose up -d

  # No need to repeat this anywhere else! It only needs to be defined once.

WORKING_DIR is a new special variable that allows us to get the directory the command was run in. Because I can now run Task from any child/sub directory, this means that I can cd services/my-service-1 and run task up and my service will be brought up by Task. This task only needs to be defined once in the root Taskfile and will work with any number of services.

Note that I've also added a precondition which will stop us from running the command on a directory that does not contain a docker-compose.yml file.

I'm sure there are also many other interesting use cases for this functionality.


A quick note on this comment regarding a security vulnerability in the same functionality of Git. I have addressed this by following the advice on the linked GitHub blog. Simply put, if a directory's ownership changes, we will stop traversing up the directory tree and report that no Taskfile has been found. You can view this specific functionality in 5fb5e70.

This is to stop potential attacks on multi-user machines when malicious code could be written in a shared location which could in turn, be accidentally run by the end user. This will not work on Windows because Go doesn't seem to have a way to get a file's uid. Not sure if there's much we can do about this besides perhaps disabling the functionality on Windows (would be a shame). If I'm honest, I'm not sure how big of a problem this really is for a project of this size (as opposed to Git which is installed on nearly every developer's system).

@pd93 pd93 force-pushed the search-for-taskfile-in-parent-directories branch from 0e06c88 to 79beb72 Compare October 31, 2022 21:47
@pd93 pd93 linked an issue Oct 31, 2022 that may be closed by this pull request
@pd93 pd93 force-pushed the search-for-taskfile-in-parent-directories branch from 79beb72 to 4a91886 Compare October 31, 2022 21:58
@pd93 pd93 force-pushed the search-for-taskfile-in-parent-directories branch from 4a91886 to 5fb5e70 Compare October 31, 2022 23:03
Copy link
Member

@andreynering andreynering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @pd93, awesome work as always! 🎉

I tested this locally and it is working flawlessly.

Before merging this, I'd like to discuss renaming WOKRING_DIR to something else, to avoid confusion with PWD (see comment).

@@ -55,6 +55,7 @@ There are some special variables that is available on the templating system:
| `TASK` | The name of the current task. |
| `ROOT_DIR` | The absolute path of the root Taskfile. |
| `TASKFILE_DIR` | The absolute path of the included Taskfile. |
| `WORKING_DIR` | The absolute path of the directory the command was run from. Use this instead of `PWD` as it will work on any OS. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use this instead of PWD as it will work on any OS.

I think WORKING_DIR is actually different from PWD? PWD will contain the directory the task in running in, and it should work even on Windows given mvdan/sh handle it IIRC.

WORKING_DIR, on the other hand, contains the directory the user is calling Task from. The idea to add this is great, but I believe that we might want to rename to something different to avoid users confounding it with PWD.

Not sure about the name, but maybe RUNNING_FROM_DIR, USER_WORKING_DIR, or something else (naming things is hard 😅 ).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just sharing ideas.. What about EXECUTION_DIR or INVOCATION_DIR?
btw. this is the feature we need to get started with taskfile :)

Copy link
Member Author

@pd93 pd93 Nov 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andreynering Apologies for the delayed reply. I was on holiday last week 🌴

I think WORKING_DIR is actually different from PWD

Yeah, the docs are definitely wrong here. I'll give this another go

it [PWD] should work even on Windows

The builtin command pwd works on Windows but PWD (the environment variable) does not.

Not sure about the name

I think USER_WORKING_DIR is sensible.

@TheKangaroo The problem with EXECUTION_DIR or INVOCATION_DIR is that it doesn't specify if its the directory where the user is executing/invoking the Taskfile, or the directory where Taskfile is executing/invoking commands. This is why I believe that something with USER in it is preferred. The naming for this is definitely a bit of a pain.

We could add TASKFILE_WORKING_DIR as well to try and clear up the distinction and just say that this is the same as PWD in the docs? This would also solve PWD not working on Windows.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed a commit changing the name so we can release this.

Thanks again @pd93!

- docker-compose up -d
```

In this example, we can run `cd <serivce>` and `task up` and as long as the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In this example, we can run `cd <serivce>` and `task up` and as long as the
In this example, we can run `cd <service>` and `task up` and as long as the

@andreynering andreynering merged commit b3627fc into master Dec 6, 2022
@pd93 pd93 deleted the search-for-taskfile-in-parent-directories branch December 6, 2022 00:58
@pd93 pd93 mentioned this pull request Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Search for Taskfiles in parent directories
3 participants