-
Notifications
You must be signed in to change notification settings - Fork 478
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cross-platform justfiles #531
Comments
Thank you for all the work your putting into cross-platform support! I think the annotations option above it a good solution that doesn't require writing your own scripting language. The update in v0.5.0 is great for people in organizations that may need to use work cross-platform and have the option of specifying organization wide to use something specific like REXX, or Python, etc. Also for individual developers that work cross-platform but don't need users to utilize the Justfile. I think it would be reasonable and legible to specify the annotation as-is affixed to the recipe name. So something like 1. Sloppy Left (current default)
2. Comments aligned
3. Three Columns
4. Stacked Annotations
5. Stacked Annotations with Comments Aligned (my favorite compact)
Number 3 is my favorite. But I always have wide terminals setup. Number 5 is my favorite for compact display. BTW, thanks for adding |
BTW, how hard would it be to just embed JavaScript, Lua, or Neko at this point? I know it was not really an option prior. |
You are most welcome!
I'm hesitant to use I definitely like the three column display for
You bet. I think powershell should work too, using
I can definitely see the value of embedding a scripting language:
The two main things that would stop me are:
|
JavaScript, Lua, Neko, and Python all have good embedding options. Lua and Neko were specifically designed with embedding in mind so have very small VMs.
Note the numbers above are just for general reference. I'm not even remotely certain of what is needed to implement any of those interpreters.
Lots of options but really for light weight and overall performance I can't see beating Neko. Though Lua is a close second with the caveats mentioned. Followed by MuJS, Duktape, etc. simply because of size. |
Thanks for this comparison! I think that a scripting language wouldn't fully obviate the need for a shell, since most people would probably want to write most recipes in shell, so an embedded scripting language would probably be a separate feature. That being said, I think Python is the only reasonable choice, since it blows the other languages away in terms of ease of shell-scripting type tasks, and it's widely known. (JavaScript is definitely widely known, but not great for scripting, and Neko and Lua are popular but still somewhat niche.) |
No prob. It's something important to me for the project and something I'm trying to learn more about in general. I love Python 3 so no problem with that. I don't know how you can implement it without doubling or tripling the size of |
I don't believe embedding makes sense as it will bloat the binary. I'll just point out for people looking here that for scripting with TypeScript/JavaScript deno really makes a lot of sense. It handles dependency fetching via normal url imports and handles TypeScript compilation. All you need to do is make sure the deno binary is in the path. For anything that requires a language runtime, it would just make sense to make the install of that a dependency of the other targets. |
I would approach this exactly as I would in Rust with In the original example: [sh]
foo:
ls /
[powershell]
foo:
dir C:\\ This should be a syntax error as the recipe names are the same (and there is no conditional attribute to select the recipe based on some precondition). There is no logical way to select which recipe to run. If there are two versions of some recipe that require different shells, I would argue that they are in fact different recipes. If this was not the case, why would they require different shells? The way to differentiate here should be in name only. By introducing a conditional attribute, recipe selection becomes automatic and is entirely in the control of the user.
I partially agree with this. I don't necessarily think it is always be desirable to use My preferred design for cross-platform justfiles would look something like: [conditional(os_family = "unix")]
[shell(sh)]
foo:
ls /
[conditional(os_family = "windows")]
[shell(powershell)]
foo:
dir C:\\ This makes it clear that:
Technically the Of course this could ultimately be extended to global variables to override defaults: [conditional(os_family = "windows")]
set shell := ["powershell.exe", "-c"]
[conditional(os_family = "unix")]
foo:
ls /
[conditional(os_family = "windows")]
foo:
dir C:\\ |
i wonder if duckscript might help you to resolve this issue.
disadvantages:
As i see it, the binary size issue is the biggest risk here. |
Duckscript looks very cool, and I definitely like the idea of embedding something like it in Just. I'm unsure about embedding something that isn't And even though |
makes sense. so I think the only solution to solve cross platform is to make sh env setup easier on windows. |
|
PowerShell would definitely be a reasonable option, although the unfamiliarity of most users is a negative. A big part of this, though, is also shipping cross-platform versions of the binaries that people call, since that's a big part of most scripts. So I'm basically waiting for a pure-rust version of something like busybox that includes a built-in |
Are you currently aware of any such projects Casey? Even if they are just starting and may not see the light of day? |
@runeimp I think the closest thing I know of is uutils/coreutils. It's a reimplementation, in rust, of much of the unix userland, and can be built into a busybox-style binary. I don't think they have a shell yet though. |
I am mostly aligned with @rjsberry but I also think I am missing something :) Personally the requirements I think of are:
For example I currently have the following sanity-tests: && sanity-tests-win
pytest sanity_tests --confcutdir sanity_tests
python modules/paths/paths_b/path.py
sanity-tests-win:
#!/usr/bin/env python
import sys, subprocess
if sys.platform == "win32":
subprocess.check_call("py -3 modules/windows_registry/test_reg2.py") I was thinking that maybe this could look like sanity-tests: && sanity-tests-win
pytest sanity_tests --confcutdir sanity_tests
python modules/paths/paths_b/path.py
[conditional(os_family = "windows")]
sanity-tests-win:
py -3 modules/windows_registry/test_reg2.py Though I haven't thought carefully whether _sanity-tests-common:
pytest sanity_tests --confcutdir sanity_tests
python modules/paths/paths_b/path.py
[conditional(os_family != "windows")]
sanity-tests: sanity-tests-common
pass # can I create an empty recipe?
[conditional(os_family = "windows")]
sanity-tests: sanity-tests-common
py -3 modules/windows_registry/test_reg2.py |
@markmmm I believe the whole point of the shell support issue is largely that Windows does not support Bourne shell ( |
One problem though that needs to be kept in mind is that even if we embed a shell in |
Example requirement: I want to delete a file (e.g. user-configuration.json) that is located in different locations on Win/Mac/Linux, on Windows I would like to additionally remove a registry entry
Embedding shell solves one part of the problem for people who want to use shell across all platforms - but it wouldn't support the full title of this issue. |
@markmmm I like the conditional logic tags. Though you can already do this with sanity-tests-common:
pytest sanity_tests --confcutdir sanity_tests
python modules/paths/paths_b/path.py
sanity-tests: sanity-tests-common
#!/bin/sh
if [ '{{os()}}' == 'windows' ]; then
py -3 modules/windows_registry/test_reg2.py
fi |
You may even be able to do it with with the conditional logic built into |
I'm using just and nushell to run cross-platform scripts, they work pretty well |
Looks good to me but I'm a PowerShell user :^) |
I've grown to appreciate PowerShell over the last year. I think that in general it's verbose as hell. But I like many of the ideas behind it. And yes, that feature in nushell is awesome! 😀 |
Yes! That's it. It's modern, elegant and powerful, I have use it for a few months. The latest version 0.40.0 is compressed |
@runeimp Try it and you will love it, I enjoy it very much! Actually, I have change all my scripts from sh to nushell. |
Yeah, it would likely need to reach version 1 before @casey would consider using it. Looks promising though. |
Regarding an embedded scripting language, have you considered rhai? It seems the easiest option to embed in Rust. |
@torkleyy rhai looks good, although the fact that it's not popular is a big of a downside, since most developers using |
i wonder if there is any more update on this support. currently, my solution is set default shell to powershell 7. |
Nothing new to report, I've been working on other projects, so I'm not tackling major new features, although if someone wants to tackle this, I'm happy to help. |
Nushell has introduced the new engine in v0.60, and just released v0.65, It's good enough for me currently. I had use it for more then half an year, and found no compatible issue for mac / Linux and Windows. |
Starlark maybe? #537 (comment) |
Now that #1387 has landed, and shells are fully configurable, I think this issue can probably be closed. Truly cross platform justfiles would depend on two things:
These two things are ultimately outside of the scope of the project. I'm hoping that eventually, they'll appear, ideally in the form of |
I know this is an older issue technically, but PowerShell core is cross-platform and has built-in # --| Example -----
if ($IsLinux) { & cp ./target/release/myapp ~/.local/bin/ }
# --| Create directory x-platform ------
# Edit -- Just tested on a Windows VM
# Running 'echo $HOME' from a pwsh.exe prompt returned 'C:\Users\<myUser>'
# So that makes it even easier to just use $HOME in scripts or other vars possibly.
$targetDir = ""
if ($IsWindows) { $targetDir = $env:HOMEPATH } # returns C:\Users\<user>\
else { $targetDir = $env:HOME } # returns /home/<user>
$myApp = "${targetDir}/.myApp"
if (!([System.IO.Directory]::Exists($myApp))) {
[system.io.directory]::CreateDirectory($myApp)
}
# This will work on any system Pwsh Core installs on, which is most any.
# (Win, Mac, Linux: Deb, Ubuntu, Rhel, Alpine, Raspi (Has Arm support), then also via Docker) So either utilizing that in a justfile, or if a justfile simply had a command annotation of #[if(linux)]
install: build
echo linux > stuff
#[if(windows)]
install: build
cmd /c myWindowCommand.stuff Not sure id that information is useful at all, but I just wanted to make sure it was known |
Yep, this is what I ended up doing previously as the workaround too. Then later I switched to nushell. I highly recommend to give it a try. The problem with Powershell Core is that on linux and mac, the first launch can be crazily slow from my experience, (as well as the second, maybe the third, but it gets better every time). And on the other side, nushell is quite fast and usually can do most of the work too, unless you need certain specific features that uses powershell modules is way easier. (don't get me wrong. I love powershell and wrote many powershell modules by myself too. : D i just don't like to wait when running frequently used commands.) |
Wow! That's good for you! Somehow pwsh needs about 1-2 minutes to launch on my work mbp, so I switched to nushell in later projects. But I am still using it on my personal fun project (w/ -noprofile and etc). And yes, I am a huge fan as well, the object pipeline and all the bash aliases just makes things so much easier and less chance to wrong. And having a fallback will be great indeed. What I am doing right now is that I have a init command for each system just to get things installed, e.g. |
@r12f you could do something like # Initialize OS Setup
init:
just _init-{{os()}}
_init-linux:
# Linux init
_init-macos:
# macOS init
_init-windows:
# Windows init |
Wow! This is a really nice trick! Thanks a lot for sharing! 👍👍👍 |
Edit: Oh... @runeimp way is much easier, lol. I got way off track from what I wanted to accomplish this evening, but I wanted to see if I could create a single command that might work for both pwsh and sh, and it looks like I have something figured out. It is in no way practical, or even worth the effort outside of my need to accomplish this, lol, but this is essentially what I came up with: shebang := if os() == 'windows' {
'pwsh.exe'
} else {
'/usr/bin/bash'
}
set shell := ["/usr/bin/env", "bash" ,"-c"]
set windows-shell := ["pwsh.exe","-NoLogo", "-noprofile", "-c"]
sh-pwsh:
#!{{shebang}}
#!/bin/bash
# ---------
# - Shell -
# ---------
echo --% >/dev/null;: ' | out-null
<#'
function ShellRunner {
echo -e "\n\nRunning: ShellRunner\n\n" || true
just install
}
# Pwsh ignores this whole block, but bash will run it
"ShellRunner"
exit #> # Bash exits here
# ---------
# - Pwsh --
# ---------
echo "`n`nRunning PwshRunner`n`n"
function PwshRunner {
$homeDir = $HOME
$originalExe = "ncm-rs.exe"
$newExe = "ncm.exe"
$targetPath = "./target/release/"
$releasePath = "./target/release/ncm.exe"
$installPath = "/.local/bin/"
# --
if (test-path $releasePath) { rm $releasePath }
$(cargo test)
$(cargo build --release)
mv "$targetPath/$originalExe" "$targetPath$newExe"
echo "Moving $releasePath to $homeDir/$installPath"
cp "$releasePath" "${homeDir}/$installPath"
}
# --------
PwshRunner 2023-02-16.22-28-51.mp4 |
@MostHated I often need cross-platform compatibility to simplify Justfile maintenance. I work on Mac but most of the systems I deploy code to are Windows so I simplified things by installing PowerShell Core on my Mac and default to PowerShell Desktop on the Windows systems as they always have that version installed. set windows-shell := ["powershell", "-c"] # To use PowerShell Desktop instead of Core on Windows
set shell := ["pwsh", "-c"] # PowerShell Core (Multi-Platform)
# The .exe extension is not required
shebang := if os() == 'windows' {
'powershell'
} else {
'/usr/bin/env pwsh'
} # Switch 'powershell' with 'pwsh' for PowerShell Core
# PS Hello World
@hello:
#!{{shebang}}
Write-Host "Shebang: {{shebang}}"
Write-Host "Hello, world from PowerShell!"
# Do other scripty things...
|
It would be useful to be able to write single justfiles that work on a variety of systems.
Concretely, justfile authors should be able to write a single justfile that can be deployed on different platforms, where the same command (i.e.
just build
) can be used to invoke the justfile and run a particular recipe or set of recipes.I just released version v0.5.0 (see #530), which allows setting the shell used to interpret backticks and recipe lines on a per-justfile basis. This allows writing justfiles that work on platforms where
sh
is not available, but it still requires writing a different justfile for each different platform.This feature has been discussed before, in particular in #326 and #161, however those issues have been addressed and closed, so I'm opening this issue to track the feature more directly.
The design which I currently favor is to add annotations to recipes that control:
cmd.exe /c
instead ofsh -cu
"sh
is not available"Example:
Since this feature is primarily intended to make just usable when
sh
is not available, it seems like different versions of the same recipe should be selected not based on which OS just is running on, but rather whether or a particular shell is available. (Since windows machines may havesh
, and ifsh
is available it is probably desirable to use it.)I suggest initially starting with three annotations,
sh
,powershell
, andcmd
. Each annotation has an associated test, which, if multiple recipes have the same name but different annotations, just will use to select which version should run.sh
sh -cu echo OK
sh -cu
cmd
cmd.exe /c Echo OK
cmd.exe /c
powershell
powershell -c "Write-Host OK"
powershell -c
Recipes with no annotation will default to
sh
, to preserve backwards compatibility.I think the above design is solid, but there are still a couple of unresolved questions:
powershell
-annotated recipe even ifsh
is available? If so how?sh
.The text was updated successfully, but these errors were encountered: