Why does globbing not work on ported Linux tools, but work on Windows ones?

[eggbean]

I know that this isn't to do with clink, as it happens without clink too, but if this does not involve readline on Linux, I don't know what it does involve.

On Linux shells and in cmd.exe when using native Windows commands, glob characters like * and ? work, but they don't when using ported Linux tools like ls, file, etc.(I am using busybox or coreutils and similar).

I thought this may be a good place to ask this. Isn't it something that clink could make work?

[chrisant996]

I know that this isn't to do with clink, as it happens without clink too, but if this does not involve readline on Linux, I don't know what it does involve.

On Linux shells and in cmd.exe when using native Windows commands, glob characters like * and ? work, but they don't when using ported Linux tools like ls, file, etc.(I am using busybox or coreutils and similar).

Short version:

In Unix/Linux the shell does it; but in Windows CMD doesn't expand wildcards.

Ported Linux apps work on Windows the same as they do on Linux. The apps don't expand wildcards. The difference in behavior happens when they aren't launched from a Linux shell.

More info:

There are lots of articles and questions/answers on this topic on the internet, findable via internet searches. Here's another. 🙃

In Unix/Linux the shells interpret wildcards, and the shells expand them before launching an app.
Typing program * turns into program foo bar qux by the time the program receives the command line.
To make the program receive the * without expansion, it must be quoted like program "*".

In CMD on Windows, the shell does not expand wildcards before an app.
Typing program * is received by the program as program *, and programs typically process wildcards themselves.

So if ported Linux apps are run from a Linux shell like bash or zsh or etc, then the shell does the wildcard expansion and the overall experience is the same as on Linux. But if the app is run from CMD, then only half of the ported experience is happening -- since shells in Linux are what expand (unquoted) wildcards, and CMD does not expand wildcards, the ported Linux apps receive the command from CMD without wildcards expanded.

By the way, here are a couple of interesting caveats/problems with the Linux shell approach to wildcard expansion:

• Suppose there are 40,000 files in a directory. Pre-expanding * into 40,000 separate filenames is inefficient. And CMD has a maximum line length of around 8000 characters.
• Expanding wildcards has interesting implications for command line syntax: an app like compare_two_files is going to behave in a reasonable but confusing way. For example a program compare_two_files *.old *.new will compare the first two *.old files in the directory (not the first *.old versus first *.new). And if there are more than 2 files matching *.old, then what happens will depend on how compare_two_files chooses to interpret input arguments -- maybe it will compare 1st/2nd and 3rd/4th and 5th/6th, etc. Or maybe it will give an error if there are more than 2 files. To get it to compare *.old files versus *.new files, you'd have to run compare_two_files "*.old" "*.new".

Linux and Windows have different paradigms for who is responsible for wildcard expansion. Neither way is "right" or "wrong", and each way has benefits and drawbacks -- they're just different ways. IIRC, the main reason Windows CMD doesn't expand wildcards is because DOS and Windows aimed at having familiarity/compatibility with CP/M (and maybe also VAX/VMS, though I don't remember offhand whether VMS expanded wildcards in its shell, and I couldn't quickly find an answer via internet searches).

Isn't it something that clink could make work?

When should it expand wildcards, and when should it not?

If it expands wildcards before launching ported linux programs, then ok fine.
If it expands wildcards before launching any other program, then it will create problems.
There isn't any reliable way to predict whether a given program is ported from Linux and expects to be launched from a Linux shell.

You could write a Lua script to expand wildcards. The key would be using clink.onfilterinput(). You can try to find a way to predict when it's reasonable to expand wildcards.

[eggbean]

Thanks for the explanation.

[chrisant996]

P.S. MSVC has an optional way to compile a program, that makes the C runtime expand wildcards in the command line before the program actually receives the command line. It can be used to make a program behave similar to as if it were launched from a Linux shell. Typically, when programs are compiled using that optional mode it creates confusion and quirky edge case problems, but the mode exists and is possible for an app author to use if they wish.

[eggbean]

Good to know. Cheers.