add non-interactive install to the install script #14

amtoine · 2023-11-14T17:52:13Z

as we've discussed @hustcer, here is an improved version of the get-latest-nightly-build command 😇

there are some minor tweaks and refactoring.
apart from that, the core of these changes are

the command will now extract and install the nu binary in --install-dir
the default is a non-interactive install
- --interactive will let the user chose among all the builds
- otherwise, a default build will be chosen depending on the architecture
- --full will install the fully-featured build of Nushell, i.e. with dataframe and extra enabled
- --musl will install the MUSL build on x86_64

errors should be given when using an invalid set of options or when something bad happens internally 🤞

toolkit.nu

hustcer · 2023-11-16T01:27:19Z

Thanks, let's give it a try and see if there is any feedback

# Description The `cell-path` is a type that can be created statically with `$.nested.structure.5`, but can't be created from user input. This makes it difficult to take advantage of commands that accept a cell-path to operate on data structures. This PR adds `into cell-path` for dynamic cell-path creation. `into cell-path` accepts the following input shapes: * Bare integer (equivalent to `$.1`) * List of strings and integers * List of records with entries `value` and `optional` * String (parsed into a cell-path) ## Example usage An example of where `into cell-path` can be used is in working with `git config --list`. The git configuration has a tree structure that maps well to nushell records. With dynamic cell paths it is easy to convert `git config list` to a record: ```nushell git config --list | lines | parse -r '^(?<key>[^=]+)=(?<value>.*)' | reduce --fold {} {|entry, result| let path = $entry.key | into cell-path $result | upsert $path {|| $entry.value } } | select remote ``` Output: ``` ╭────────┬──────────────────────────────────────────────────────────────────╮ │ │ ╭──────────┬───────────────────────────────────────────────────╮ │ │ remote │ │ │ ╭───────┬───────────────────────────────────────╮ │ │ │ │ │ upstream │ │ url │ git@github.com:nushell/nushell.git │ │ │ │ │ │ │ │ fetch │ +refs/heads/*:refs/remotes/upstream/* │ │ │ │ │ │ │ ╰───────┴───────────────────────────────────────╯ │ │ │ │ │ │ ╭───────┬─────────────────────────────────────╮ │ │ │ │ │ origin │ │ url │ git@github.com:drbrain/nushell │ │ │ │ │ │ │ │ fetch │ +refs/heads/*:refs/remotes/origin/* │ │ │ │ │ │ │ ╰───────┴─────────────────────────────────────╯ │ │ │ │ ╰──────────┴───────────────────────────────────────────────────╯ │ ╰────────┴──────────────────────────────────────────────────────────────────╯ ``` ## Errors `lex()` + `parse_cell_path()` are forgiving about what is allowed in a cell-path so it will allow what appears to be nonsense to become a cell-path: ```nushell let table = [["!@$%^&*" value]; [key value]] $table | get ("!@$%^&*.0" | into cell-path) # => key ``` But it will reject bad cell-paths: ``` ❯ "a b" | into cell-path Error: nu::shell::cant_convert × Can't convert to cell-path. ╭─[entry #14:1:1] 1 │ "a b" | into cell-path · ───────┬────── · ╰── can't convert string to cell-path ╰──── help: "a b" is not a valid cell-path (Parse mismatch during operation.) ``` # User-Facing Changes New conversion command `into cell-path` # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting Automatic documentation updates

# Description  This PR adds type checking of all command input types at run-time. Generally, these errors should be caught by the parser, but sometimes we can't know the type of a value at parse-time. The simplest example is using the `echo` command, which has an output type of `any`, so prefixing a literal with `echo` will bypass parse-time type checking. Before this PR, each command has to individually check its input types. This can result in scenarios where the input/output types don't match the actual command behavior. This can cause valid usage with an non-`any` type to become a parse-time error if a command is missing that type in its pipeline input/output (`drop nth` and `history import` do this before this PR). Alternatively, a command may not list a type in its input/output types, but doesn't actually reject that type in its code, which can have unintended side effects (`get` does this on an empty pipeline input, and `sort` used to before #13154). After this PR, the type of the pipeline input is checked to ensure it matches one of the input types listed in the proceeding command's input/output types. While each of the issues in the "before this PR" section could be addressed with each command individually, this PR solves this issue for _all_ commands. **This will likely cause some breakage**, as some commands have incorrect input/output types, and should be adjusted. Also, some scripts may have erroneous usage of commands. In writing this PR, I discovered that `toolkit.nu` was passing `null` values to `str join`, which doesn't accept nothing types (if folks think it should, we can adjust it in this PR or in a different PR). I found some issues in the standard library and its tests. I also found that carapace's vendor script had an incorrect chaining of `get -i`: ```nushell let expanded_alias = (scope aliases | where name == $spans.0 | get -i 0 | get -i expansion) ``` Before this PR, if the `get -i 0` ever actually did evaluate to `null`, the second `get` invocation would error since `get` doesn't operate on `null` values. After this PR, this is immediately a run-time error, alerting the user to the problematic code. As a side note, we'll need to PR this fix (`get -i 0 | get -i expansion` -> `get -i 0.expansion`) to carapace. A notable exception to the type checking is commands with input type of `nothing -> <type>`. In this case, any input type is allowed. This allows piping values into the command without an error being thrown. For example, `123 | echo $in` would be an error without this exception. Additionally, custom types bypass type checking (I believe this also happens during parsing, but not certain) I added a `is_subtype` method to `Value` and `PipelineData`. It functions slightly differently than `get_type().is_subtype()`, as noted in the doccomments. Notably, it respects structural typing of lists and tables. For example, the type of a value `[{a: 123} {a: 456, b: 789}]` is a subtype of `table<a: int>`, whereas the type returned by `Value::get_type` is a `list<any>`. Similarly, `PipelineData` has some special handling for `ListStream`s and `ByteStream`s. The latter was needed for this PR to work properly with external commands. Here's some examples. Before: ```nu 1..2 | drop nth 1 Error: nu::parser::input_type_mismatch × Command does not support range input. ╭─[entry #9:1:8] 1 │ 1..2 | drop nth 1 · ────┬─── · ╰── command doesn't support range input ╰──── echo 1..2 | drop nth 1 # => ╭───┬───╮ # => │ 0 │ 1 │ # => ╰───┴───╯ ``` After this PR, I've adjusted `drop nth`'s input/output types to accept range input. Before this PR, zip accepted any value despite not being listed in its input/output types. This caused different behavior depending on if you triggered a parse error or not: ```nushell 1 | zip [2] # => Error: nu::parser::input_type_mismatch # => # => × Command does not support int input. # => ╭─[entry #3:1:5] # => 1 │ 1 | zip [2] # => · ─┬─ # => · ╰── command doesn't support int input # => ╰──── echo 1 | zip [2] # => ╭───┬───────────╮ # => │ 0 │ ╭───┬───╮ │ # => │ │ │ 0 │ 1 │ │ # => │ │ │ 1 │ 2 │ │ # => │ │ ╰───┴───╯ │ # => ╰───┴───────────╯ ``` After this PR, it works the same in both cases. For cases like this, if we do decide we want `zip` or other commands to accept any input value, then we should explicitly add that to the input types. ```nushell 1 | zip [2] # => Error: nu::parser::input_type_mismatch # => # => × Command does not support int input. # => ╭─[entry #3:1:5] # => 1 │ 1 | zip [2] # => · ─┬─ # => · ╰── command doesn't support int input # => ╰──── echo 1 | zip [2] # => Error: nu:🐚:only_supports_this_input_type # => # => × Input type not supported. # => ╭─[entry #14:2:6] # => 2 │ echo 1 | zip [2] # => · ┬ ─┬─ # => · │ ╰── only list<any> and range input data is supported # => · ╰── input type: int # => ╰──── ``` # User-Facing Changes  **Breaking change**: The type of a command's input is now checked against the input/output types of that command at run-time. While these errors should mostly be caught at parse-time, in cases where they can't be detected at parse-time they will be caught at run-time instead. This applies to both internal commands and custom commands. Example function and corresponding parse-time error (same before and after PR): ```nushell def foo []: int -> nothing { print $"my cool int is ($in)" } 1 | foo # => my cool int is 1 "evil string" | foo # => Error: nu::parser::input_type_mismatch # => # => × Command does not support string input. # => ╭─[entry #16:1:17] # => 1 │ "evil string" | foo # => · ─┬─ # => · ╰── command doesn't support string input # => ╰──── # => ``` Before: ```nu echo "evil string" | foo # => my cool int is evil string ``` After: ```nu echo "evil string" | foo # => Error: nu:🐚:only_supports_this_input_type # => # => × Input type not supported. # => ╭─[entry #17:1:6] # => 1 │ echo "evil string" | foo # => · ──────┬────── ─┬─ # => · │ ╰── only int input data is supported # => · ╰── input type: string # => ╰──── ``` Known affected internal commands which erroneously accepted any type: * `str join` * `zip` * `reduce` # Tests + Formatting  - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting  * Play whack-a-mole with the commands and scripts this will inevitably break

amtoine added 9 commits November 14, 2023 18:41

extract and install the binary

baf53e4

add --interactive

4b1ac38

refactor dump dir in variable

baae797

start implementing OS non-interactive defaults

97f0685

add log statements

263f79e

do not consider MSI builds

7120b60

implement OS mapping with --full and --musl

aab2a2d

fix the --full check

6180e49

rewrite the dosctring of the command

f9edb46

amtoine requested a review from hustcer November 14, 2023 17:52

hustcer reviewed Nov 15, 2023

View reviewed changes

toolkit.nu Show resolved Hide resolved

toolkit.nu Show resolved Hide resolved

hustcer merged commit 22100e1 into nushell:nightly Nov 16, 2023

amtoine deleted the non-interactive-install branch November 16, 2023 17:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add non-interactive install to the install script #14

add non-interactive install to the install script #14

Uh oh!

amtoine commented Nov 14, 2023

Uh oh!

Uh oh!

Uh oh!

hustcer commented Nov 16, 2023

Uh oh!

Uh oh!

add non-interactive install to the install script #14

add non-interactive install to the install script #14

Uh oh!

Conversation

amtoine commented Nov 14, 2023

Uh oh!

Uh oh!

Uh oh!

hustcer commented Nov 16, 2023

Uh oh!

Uh oh!