-
Notifications
You must be signed in to change notification settings - Fork 481
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modules and subcommands #383
Comments
Thanks for opening this more focused issue Currently for us, it would be enough to support one level deep submodules. But of course, it would be nice if the solution we could come up with would work for deeper submodules, too. I'm not sure if # <root>/Justfile
mod foo
# <root>/foo/mod.just
test:
echo "All OK" And then you could call This way # <root>/Justfile
mod foo
# <root>/foo/mod.just
mod bar
# <root>/foo/bar/mod.just
RECIPE:
echo "All OK" I'm not excited about naming sub-modules with # <root>/Justfile
mod foo "tasks/foo/Justfile"
# <root>/tasks/foo/Justfile
RECIPE:
echo "All OK" which could be called like so:
One could continue with the same way to enable sub-submodules:
and then that could be called with something like this:
It could be that overloading the directory syntax is not wise, as # <root>/foo/Justfile
RECIPE:
echo "All OK"
The syntax isn't the main point for sub-submodules. One possibility would be to denote sub-submodules with e.g.
I'd be fine with that kind of solution, too. What do you think? What kind of discussion would you want to have here? I'd be happy to continue the conversation in any way you'd want to |
I'm just going respond like I didn't let your reply in my inbox for a month >_< I beg your forgiveness!
Agreed, it would be nice to do something that works with deeper submodules. I've heard tell that some companies use
This sounds good.
Yeah, I definitely want to avoid globbing.
We should come up with something that lets
I think using a string as in I was thinking that given
Here is great! I promise I'll respond faster next time 😅 |
One thing that needs to get resolved: Can a recipe in one module have dependencies on a recipe in another module? And if so, how do they refer to recipes in other modules? Is this something that you need or would use? Also if there are others out there who would make use of modules, I'd definitely be interested in your thoughts. |
Hey, it's completely OK to take your time to reply, or to not even reply at all
Hmm yeah, we can make the location optional for sure — but I'm not sure if we know yet what would be a good choice from all of these, so I'm a bit weary on making the decision already in the first implementation. I kinda like the name I'm worried about folder bloat if this default were any of the three you suggested. If, on the other hand, modules were to live inside one directory deeper first, then I don't have a strong feeling on any of the suggestions. What I mean is something like
This could work — it would also be possible to give a helpful error should someone have this kind of mod foo
foo:
echo "hi" and saying something like this:
That way we wouldn't even have to figure out any special syntax 🙌
Hoo boy, we're going to the dependency management domain 😅. If at all possible, I think we should avoid making any decisions on automatic behavior yet, in order to see how submodules would be used on their own. If one would need to share recipes, one could refer to a module by path, e.g. by climbing up the tree: # <root>/Justfile
mod foo
hello:
foo bar greet World
# <root>/just_modules/foo/Justfile
mod bar "../bar/Justfile"
# <root>/just_modules/bar/Justfile
greet subject:
echo "Hello {{subject}}" Then So if one would have something like this: # <root>/Justfile
mod foo
hello:
foo bar greet World
# <root>/just_modules/foo/Justfile
mod bar "../bar/Justfile"
# <root>/just_modules/bar/Justfile
mod foo "../foo/Justfile"
greet subject:
foo bar greet "Hello {{subject}}"
|
Aww, thanks 😊
Yeah, definitely. Hmm, so maybe the best thing to is to wait until we have more experience with the feature, before picking a default location or locations that just should look for submodules. If we do the explicit-path form, as you suggested:
Then we can wait and see which, if any, default location make sense, based on user feedback
I think you have a good point about folder bloat, and we can use the explicit-path form at first to avoid committing to anything.
I think that's a good idea. One thing that I've learned from writing If you allow a construct, it's annoying to deprecate it and can be dangerous to suddenly change its behavior in a way that users don't expect. However, if you didn't allow it in the first place, adding it later is very easy.
Yeah, good idea. This stuff is why the idea of adding submodules has kind of freaked me out, so it's probably best to implement a true MVP first, and then expand as necessary.
And cyclical dependencies in variable assignments:
That sounds good. Static analysis for the win 🧐 |
Great! I guess we're now in the "plan is ready, time to start coding" phase 😅. My rust skills are, err..., rusty — I have been meaning to get to know rust a bit but I haven't had the time. Is there any way I could help even though I would not be able to implement this? Again, even if this would not be implemented, that would also be OK |
I like where this seems to be going for MVP. One addition I'd really appreciate is the ability to point to a module via
I would be extremely happy. 😃 |
@runeimp do you mean that |
Fortunately, before needing to worry about implementing it, we can continue to noodle on the design! I've been thinking that using strings to represent paths might be problematic. Paths differ on different platforms, e.g. unix and windows:
If module paths are strings, without some extra compatibility mechanism, justfiles will not be portable across platforms. A couple ideas to resolve this:
I like 2 a bit better, but it comes with problems of its own. If modules are required to be identifiers, are spaces and special characters allowed? |
Yay! 😄
Might be, yes
I like this approach better than a custom syntax. This would map to how e.g. If we were to take any other choice than using I am only speaking from my experiences, though. But from my point of my view, I would expect |
@valscion I meant to point to $HOME so I can target common module(s) I will use on all systems I work on. So to be clear either |
@casey Actually Windows has supported POSIX/UNIX style paths since Windows XP. And in DOS prior to that if I remember correctly. It just prefers |
Thanks for clarifying, @runeimp |
I tried it out, and it looks like windows doesn't support |
I wish I had time to dedicate to this, but it'll have to go on the back burner for now. It's a big project, with lots of implications codebase-wide, but if someone is interested in picking this up, I can provide some pointers. |
@casey Actually, |
I'm still thinking about this and definitely want to do it. I landed a much improved lexer, and I'm rewriting the parser to better separate the actual parsing (turning a token stream into higher-level structures) and consistency checks (duplicate variable names, circular dependencies, etc). Both will make extensions like these easier. I think the way that I can best move forward with this is to start with inline modules. From rust, for example: // inline module
mod foo {
fn bar() {}
}
// out of line module, loaded from `baz.rs` or `baz/mod.rs`
mod baz; By starting with inline modules, we can nail down the semantics of submodules and subcommands, without having to think about loading multiple files from disk, which will be a separate complication. We could lift rust's syntax:
Another option is to use whitespace:
Inter-module paths would be
I think that cc: @cledoux, since he was interested in including other justfiles, possibly conditionally, which this could be extended to. By adding some kind of conditional activation of modules,
|
I'm leaning towards using something with Also thinking about "block recipes", which introduce an anonymous scope just for that recipe, for local variables:
|
Hi @casey, I might have some free cycles coming up. Do you have a "current" design/expected syntax on what's wanted for this feature? |
Hi @ethankhall! I'm still pretty uncertain about the syntax. I think the way to go is pick some syntax, implement modules with that syntax, and it behind a flag. Then we can bikeshed the syntax and details of the semantics after we have more experience with the feature. Also, we should implement in-line modules first, and then later implement out-of-line modules (i.e. modules in another source file) since that's more complex. Some syntax possibilities for a module curly braces
double colon
keyword + colon
I'm leaning towards the double colon syntax:
Out-of-line modules can just omit the module suite:
Or provide a path to the file to load:
But that can be done later. How does that all sound? Please let me know if you think there are particularly strong reasons to prefer another syntax. |
This will likely be a pretty complex feature, and I'm very happy to provide as much support as needed to implement it! |
Sounds good to me. I will look over the code base and see what I can find. I'll come up with an impl plan and post it here, or rough PR with my thoughts. I'll reach out with questions! |
Sounds good! In general, parsing and consistency analysis is pretty gross, so feel free to shoot me any questions about it. |
I agree its weird out of context. Just pushed step one. I have the flag added to the config object and the help test modified to pass with the new option. Next step is to add the business logic to the analyzer and a few other files. |
@MadBomber Sounds good! Feel free to push a WIP draft PR if you want to get early feedback or have questions about anything. |
Hmm I wonder if #1095 was supposed to entirely close this issue? |
Ah, sorry, no I definitely didn't meean to close this issue |
Cool project - this import sounds very interesting. I had 2 questions: Thanks - very great work. I love how many people create cool CLI tools in Rust these days! |
@hhoeflin I've used M4 before to do similar things - docker compose junk. This time I wanted to see how close I could come with a Ruby/Crystal implementation that shared a common code base. Still not there. Like everyone else time is in short supply these days. The |
@hhoeflin I don't think reading a justfile from stdin would be hard, exactly, but it might be a tricky PR, since it would require modifying a lot of error messages. Feel free to open an issue! |
Thank you, but looking at this https://serverfault.com/questions/398514/pass-a-pipe-to-a-command-that-expects-a-filename I learned it is not actually necessary and there is a good workaround, at least on linux in bash. Always something new to learn :) |
As I learn about So much so that it seems that The one thing I've looked for and not seen is GNU-make style includes, though it seems like there are a few related discussions. I'd suggest that the ideas of modules and namespace could be made more independent from each other. Both are useful but could perhaps be implemented with some optional interdependence. In fact, the ability to compose recipes from modular Makefiles based on path is one of most successful and comfortable patterns I've seen emerge. For my use-cases I've taken to adding an It's very useful to be able to conceptually organize content while avoiding the disadvantages of recursive calls. It helps to keep the size of each individual file to a reasonable minimum, as well as making it easy to disable a module by renaming it from I trust the course of this issue and just wanted to put forward the merits of this approach since it's something that's worked well for me that I can happily recommend. |
Interesting idea to make the include path glob'able. Current |
@nickgarber I just released v1.2.2 of justprep a stop-gap pre-processor for just utility that adds the path globing capability. It works in both the Ruby gem and Crystal binary implementations. I took this issue as the excuse to refactor the code base to extract common methods into their own files for use by both Ruby and Crystal. I added the file extension ".crb" to those files which can run under both implementations. I originally thought of ".crb" to meaning Crystal and Ruby; but, now I've come closer to thinking of it as "compiled ruby." |
Hi @casey, firstly this is an outstanding tool! Kudos. Secondly, is there any update on the progress or idea when this feature may be worked on? I appreciate you're busy and have a life outside of GH 😄 - but submodules would be a killer feature to have 😄 |
@SteveChurch There is a "fake module" convention that works in most circumstances. I documented in the wiki for the pre-processer https://github.com/MadBomber/justprep/wiki/Fake-Module-Convention |
HI @SteveChurch, thanks for checking in! I actually don't have a life outside of GH 😅 but I'm currently working on another project, ord, that's getting a huge amount of attention, so it's taking up all spare bandwidth. No updates to report on modules, but check out include directives which, depending on your use-case, might be useful. |
Thanks a lot for the link! However, I haven't found a solution yet to pass arguments, have you heard of any? E.g. I try to call just myModule myRecipeInModule myArgument
// inside the module file:
myRecipeInModule myArgument="defaultValue":
@echo {{myArgument}} I've also tried to Note I've also seen this Interactive Chooser section in the docs. Maybe it's useful for other solutions to implement "fake" modules? Edit: To clarify my goal a bit: We have a big project and try to use just with command groups. The solution you provided here, allows us -- in principle -- to have a root docker recipe="help":
just -f ./commands/docker.justfile {{recipe}} Right now, it's just passing the arguments that is problematic for us. |
Ok, in the meantime, I found a solution for my "problem" using variadic parameters. You may find this helpful (or not ;) # ./justfile
docker recipe *ARGS:
just -f ./commands/docker.justfile {{recipe}} {{ARGS}} # ./commands/docker.justfile
# List all available Docker commands.
help:
@just --list --justfile {{justfile()}}
alias s := shell
# Puts you into a shell of your desired development docker container.
shell name="myDefaultContainer":
docker exec -it $(docker ps -qf "name=development-{{name}}") bash |
Yeah this is basically the same solution we've been quite happy with since the beginning! The only downside of this is that if you have special characters (such as whitespaces or quotes), then passing them down successfully to the downstream justfiles gets quite complex. You might want to try the positional arguments as per the comment by @casey here: I haven't tried that fully yet myself. |
@valscion Thanks, will keep that in mind and come back to it if/once we encounter that situation 👍 |
IT HAS BEEN NIGH ON FIVE YEARS, BUT JUST FINALLY HAS MODULES. I just released 1.19.0, which includes modules. See the readme for more information, but in short, modules can be defined using a There are currently a bunch of limitations and missing features, so modules are unstable for now. See #929 for more details. Thanks so much for everyone's copious patience! And special thanks to @neunenak, who laid a bunch of the ground work which enabled this. If this doesn't work for your use case, please feel free to open an issue detailing what doesn't work. |
This is apropos of @valscion's comment in #208, where he talks about some of the annoyances of using
just
to emulate subcommands. This is also somewhat related to #237 and #367, where sub-justfiles are discussed.I'd like to come up with something that enables all of these things. Some ideas:
A statement
mod foo
parsesfoo.just
orfoo/mod.just
as a submodule. Recipes infoo.just
/foo/mod.just
can now be called on the command line withjust foo RECIPE
.If the submodule source file is
foo.just
, the current working directory for recipes in that submodule will be the current directory. If the it is infoo/mod.just
, the current working directory will befoo/
.Questions:
What do we do about directories with weird characters? I think punting is reasonable. But eventually we might want to support something like
mod "path with spaces/bar"
. Any solution should be based on user feedback.How do we handle dependencies between recipes and variables in different submodules?
Can you have a submodule in a submodule, and call those commands?
Notes for a first path for out-of-line modules only:
The text was updated successfully, but these errors were encountered: